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Abstract. In this paper we show that the multiphcities of holomorphic dis- 
crete series representations relatively to reductive subgroups satisfy the credo 
"Quantization commutes with reduction" . 
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1. Introduction 

The orbit method, introduced by KiriUov in the 60's, proposes a correspondence 
between the irreducible unitary representations of a Lie group G and its orbits 
in the coadjoint representation : the representation ttq should be the geometric 
quantization of the Hamiltonian action of G on the coadjoint orbit O C g*. The 
important feature of this correspondence is the functoriality relatively to inclusion 
H ^ G oi closed subgroups. It means that if we start with representations ttq and 
ttq, attached to the coadjoint orbits O C Q* and C C f)*, one expects that the 
multiplicity of ttq, in the restriction ttq\h can be computed in terms of the space 

(1) Onn~liO')/H, 

where 7r(,^g : g* — >■ f)* denotes the canonical projection. The symplectic geometers 
recognise that ([1]) is a symplectic reduced space in the sense of Marsden-Weinstein, 
since TTfj^g : O — > ()* is the moment map relative to the Hamiltonian action of H on 
O. Let us give some examples where this theory is known to be valid. 

For simply connected nilpotent Lie groups, Kirillov j24j described the correspon- 
dence O I— >■ ttq, and Corwin-Greenleaf [10] proves its functoriality relatively to 
subgroup : the multiplicity appearing in the direct integral decomposition of ttqI// 
is the cardinal of the reduced space ^ . 

For compact Lie group, G. Heckman |21] proved that the multiplicity was asymp- 
totically given by the volume of the reduced space ([1]). Just after Guillemin and 
Sternberg |19j replaced this functoriality principle in a more geometric framework 
and proposed another version of this rule for a good quantization process: the 
quantization should commute with the reduction. This means that if Q//(M) is the 
geometric quantization of an Hamiltonian action of a compact Lie group H on a 
symplectic manifold M, the multiplicity of the representation tt^, in Qh{M) should 
be the (dimension of the) quantization of the reduced space ($ff)~^(C'')/if. Here 
$^ : Af — > ti* denotes the moment map. 

A good quantization process for compact Lie group action on compact symplectic 
manifolds turns to be the equivariant index of a Dolbeault-Dirac operator [JTJ 33] . 
In the late 90's, Meinrenken and Meinrenken-Sjamaar proved that the principle of 
Guillemin-Sternberg works in this setting [3TJI32. Afterwards, this quantization 
procedure was extended to non-compact manifolds with a proper moment map by 
Ma-Zhang and the author [36l [29l |39] . See also the recent work of Duflo-Vergne on 
the multiplicities of the tempered representations relatively to compact subgroups 

The purpose of this article is to show that the quantization commutes with re- 
duction principle holds in a case where the group of symmetry is a real reductive 
Lie group. Loosely speaking, we prove that if ttq and ttq, are holomorphic discrete 
series representations of real reductive Lie groups H C G, the multiplicity of ttq, 
in the restriction ttqIh is equal to the quantization of the reduced space ([1]). 

We turn now to a description of the contents of the various sections, highlighting 
the main features. 

In Section [21 we clarify previous work of Weinstein [46] and Duflo- Vargas [15[[T6] 
concerning the Hamiltonian action of a connected reductive real Lie group G on a 
symplectic manifold M . The main point is that if the action of G on M is proper 
and the moment map : M — >■ g* relative to this action is proper, then the image 
of is contained in the strongly elliptic open subset g*g and the manifold has a 
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decomposition 

(2) il/ = G X K Y. 

Here if is a maximal compact subgroup of G, and y is a closed if-invariant sym- 
plectic sub-manifold of M. Thanks to (l2|), we remark that the reduced space 
($^j^)~^(0)/G is connected for any coadjoint orbit O C g* : this is a notable 
difference with the nilpotent case where the reduced space ^ can be disconnected. 

The decomposition ^ will be the main ingredient of this paper to prove some 
quantization commutes with reduction phenomenon. Note that P. Hochs already 
used this idea when the manifold Y is compact to get a quantization commutes with 
reduction theorem in the setting of KK-theory |22) . Hochs was working on some 
induction process, while we will use ([2]) to prove some functoriality relatively to a 
restriction procedure. 

In this context, it is natural to look at the induced action of a reductive subgroup 
G' C G on Af, and we know then that we have another decomposition M = 
G' Xk' Y' if the moment map is proper. In Section [^31 we give a criterion 
that insures the propcrncss of . 

In Section [3l we turn onto a closed study of the holomorphic discrete series 
representations of a reductive Lie group G. Recall that the parametrization of 
these representations depends on the choice of an element z in the center of the 
Lie algebra of K such that the adjoint map ad(z) defines a complex structure on 
g/f. Let T be a maximal torus in K , with Lie algebra t. The existence of element 
z forces t to be a Cartan sub-algebra of g, and it defines a closed cone C{Joi(-^) 
If A* C t* is the weight lattice, we consider the subset 

GhoiW :=CLi(^)nA; 

where is the set of dominant weights. The work of Harish-Chandra tells us 
that that we can attach an holomorphic discrete series representation to any 
A e Ghoi(z)- 

In Section [3l we look at formal quantization procedures attached to the Hamil- 
tonian action of G on a symplectic manifold M . We suppose that the properness 
assumptions are satisfied and that the image of the moment map is contained 
in G • C g*g. Let us briefly recall the definition. We define the formal 

geometric quantization of the G-action on M as the following formal sum 

(3) Qg^{M):= QiMx^G)V^, 

AeGhoi(2) 

where Q(Ma,g) G Z is the quantization of the compact symplectic reduced space 

Since the moment map $^ is proper, we can also define the formal geometric 
quantization of the JiT-action on M as 

(4) Qk°°(M) J2 V,^, 

where M^^k '■= {^^m)^^{K ■ tJ-)/K, and V"^^ denotes the irreducible representation 
of K with highest weight fi. The formal quantization procedure Qx°°, together 
with its functorial properties, has been studied by Ma-Zhang and the author in 
[361E91I39]. 
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Let R °°{G, z) be the Z-module formed by the infinite sums X^asGi i(z) "^-^ 
with m\ E Z. We consider also the Z-module R~°°{K) formed by the infinite sums 
SpGA* ""M ^iJ^ with 71^ S Z. The following basic result will be an important tool 
in our paper (see Lemma l3.20p . 

Lemma A We have an injective restriction morphism vk.g ■ R^°°{G, z) — )■ 
R-'^{K). 

We can state one of our main result 

Theorem B // Assumption^ Al or A2 are satisfied, we have the following 
relation 

The main tool for proving Theorem B, is the following relation 

where QJ^°°{Y) is the formal geometric quantization of the slice Y and 5'(p) is the 
symmetric algebra of the complex /iT-module p := (g/£, ad(z)). 

Consider now a connected reductive subgroup G' <Z G such that its Lie algebra 
g' contains the element z. Let ^q.x be the moment map relative to the Hamiltonian 
action of G" on the coadjoint orbit G • A, A e Ghoi(^)- It is not difficult to see that 
^Q.x is a proper map. Thanks to the work of T. Kobayashi [27] and Duflo- Vargas 
jl5j . we know that the representation admits an admissible restriction to G'. It 
means that the restriction V^\gi is a discrete sum formed by holomorphic discrete 
series representations , ^ G G'^^^{z). 

We can now state the major result of this paper (see Theorem 13. 26|) . 

Theorem C Let A G G\io\{z). We have the following relation 

v^\G' = QGr{G-\). 

It means that for any /i g GJ^qj(z), the multiplicity of the representation in the 
restriction V^\g' is equal to the geometric quantization Q((G • A)^_g') eTL of the 
(compact) reduced space (G • A)^_g'. 

In Section [3. 7i we prove that the formal quantization process Qq°° is functorial 
relatively to reductive subgroups. 

Theorem D Suppose that Assumption A2 holds. Then we have 

vg'..g{Qg°"{M)) = Q-^^{M). 

where Vqi^q is a restriction morphism. 

In |23j . Jakobsen-Vergne proposed another formula for the the multiplicity of 
the representation 1/p in the restriction V^\g'- In Section [?751 we explain how to 
recover their result from Theorem B. 

Section [4] is devoted to the proofs of the main results of this paper. We use 
here previous work of the author on localization techniques in the the setting of 
transversally elliptic operators. 



See Assumptions 13.23] 
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Notations. In this paper, G will denoted a connected real reductive Lie group. 
We take here the convention of Knapp [26]. We have a Cartan involution Q on G, 
such that the fixed point set K := is a connected maximal compact subgroup. 
Wc have Cartan decompositions : at the level of Lie algebras q — t@ p and at the 
level of the group G ~ /v x cxp(p). Wc denote by & a G-invariant non-degenerate 
bilinear form on g that defines a /^-invariant scalar product (X, Y) :— —b{X, Q{Y)). 

When V and V' arc two representations of a group H, the multiplicity of V in 
V will be denoted [V : V']. 

2. HAMILTONIAN actions of real REDUCTIVE LlE GROUPS 

This section is mainly a synthesis of previous work by Weinstein |46j . Dufio- 
Vargas [151 HI] and Hochs [22] , except the criterion that we obtain in Section 12.31 

Let G be a connected real reductive Lie group. We consider an Hamiltonian ac- 
tion of G on a connected symplectic manifold (M, ilm)- The corresponding moment 
map : M — > g* is defined (modulo a constant) by the relations 

(5) L{XM)nM = -d{<i>f,,X), VXeg, 

where Xm(to) := j^e~^'^ ■ m\s=o is the vector field generated by X G g. 

Let g = t © f) be a Cartan decomposition. Let K C G he the maximal compact 
subgroup with Lie algebra t. Thus we have a decomposition 

= '^M © 

where : M t* is a moment map relative to the action of K on (Af, V.^/), and 
: M — )• p* is /^-equi variant. 

The X-invariant scalar product = —b{X,Q{X)) on g induces an identifi- 

cation C I, fl* — defined by {^,X) := {^,X) for ^ e Q* and X e g. We stih 
denote := the corresponding scalar product on g*. 

Let k'~^,k^ and be respectively the Hamiltonian vectors fields of the K- 
invariant functions ^\\^m\\^, ^H^mIP; and ^||$mI|^. The relations (O give 
that 

(6) K~{m) 
for - e {G,i^,p}. 



(m), Vm G Af, 

M 



2.1. Proper actions. In this section we suppose CI : the action of G on M is 
propeJl. We have then the fundamental fact. 

Lemma 2.1. • The map : M — > p* is a K-equivariant submersion, so for any 
a e p*, the fiber Ya := ($^^)~^(a) is either empty or a sub-manifold of M . 
• The set of critical points of \\<^\[\\'^ : M ^ M is Yo := ($P^)-i(0). 

Proof. Let us prove the first point. Let m e M . Since the tangent map T$^^(m) : 
TmAf p* satisfies 

(7) {T<^l,{m),X) =-i{XM)^MU, VXep, 

the orthogonal of the image of T$^j(to) is equal to p„i := {X e p | Xj\/(to) = 0}. 
As the action of G on M is proper, the stabilizer subgroup G,„ is compact. This 
forces p„i = Lic(G„i) n p to be reduced to {0}. Thus T$^j(m) is onto and the first 



^For any compact subset A of M the subset {g S G [ g ■ A n A ^ 0} is compact. 
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Table 1 . Strongly elliptic set 



G 




GL(n,C) 


no 


0(ri,C) 


no 


SL(n,R) 


n = 2 


SOo(p,g) 


pq even 


Sp(?i,M) 


yes 


S0*(2n) 


yes 


U(p,g) 


yes 


Sp(p, q) 


yes 



point is proved. Let m S M be a critical point of The Hamiltonian vector 

field vanishes at m, and ^ tells you that $^(m) e pj„ = {0}. The second 
point is proved. □ 

For the remaining part of this section, we consider the i^-invariant sub-manifold 

r Fo C M 

that we suppose non empty. Let us consider the restriction fiy of the symplectic 
structure VLm to Y. For y e F, let p • y = {Xa/(j/), X e p} C TyM. The tangent 
space TyY is by definition the kernel of T$p(?/). Relations ^ show that 

(8) T,y ^ (p • y)^ 

where the orthogonal is taken relatively to the symplectic form. Hence the kernel 
of ^lY\y is equal to (p • y)^ f^p ■ y. For X, X' G p and y G F, we have 

= ([<(y),X],X'). 

Hence (p • y)-*- n p • y ~ n p for ^ = ^^(y). Note that for ^ e I, we have 
0? = 0j n 4 © 0^ n p. We have then proved 

Lemma 2.2. Let y Cz Y. The 2- form flyly is non degenerated if and only C ? 
for^ = <i>fj{y). 

We have a canonical G-equivariant map tt : G Xk Y — > M that sends [g, y] to 
5 • y. Following Weinstein |46j . we consider the G- invariant open subset 

(9) 0*g = G 0* I G^ is compact} 
of strongly elliptic elements. 

Example 2.3. For the group G = SL2(R), the set g*^ is equal, through the trace, 
to the cone {X e s^W | det(X) > 0}. 

Let t be the Lie algebra of a maximal torus in K. Weinstein proves that the 
open subset g*g is non-empty if and only if t is a Cartan sub-algebra of g [JB]. See 
the Table m We note that fi*^ := g*^ n I* is equal to G T | G^ C K} and that 

(10) g:,=Ad*(G).«:,. 

Let us consider the invariant open subsets Mse — {^^i)^"^ {Q*se) ^ M and Yse ■— 
Y n M,e C Y. 
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Lemma 2.4. • The 2-form is non-degenerated on Yge- 

• The action of the group K on {Ysei^Y^^) is Hamiltonian, with moment map 
$y that is equal to the restriction of <f>^j to Yge ■ 

• The map ir induces a G-equivariant diffeomorphism iTse '■ G Xk Y^e — >■ Mse- 

Proof. The first point is a direct consequence of Lemma 12.21 The second point is 
immediate. Let us check the last point. 

Relation ([10]) shows that Tr^e is onto. Let [go, 2/o], [giiVi] such that ga-ya = gi -yi- 
Then by taking the image by the moment map, we get Ad* (5o)Co = Ad* where 
the = ^MiVk) belong to Let h = g^^ga G G. We have Ad*(/i)^o = 6, and 
Ad*(0(/i))^o = Ci by taking the Cartan involution. Finally h~^Q{h) S G^g. Since 
G^o C A', we get that h € K, and finally that [50, 2/o] — [ffi, yi] in G Yse- □ 

Let us denote by r^M^ t^e restriction of the symplectic form J7m on the open 
subset Mse- We will now finish this section by giving a simple expression of the 
pull-back 7r*jnM,J eA^{Gx k Yse)- 

Let 9^ S -^^(G) (8) be the canonical connexion 1-form relative to the G-action 
by right translations : L{X'-)e^ = X,\/X G g, where X'-(g) = f^{ge*^)\Q. Let 
9^^ G .4^ (G) (g) 6, the composition of 9'-^ with the orthogonal projection X 
from to 1. We will use the G x if- invariant 1-form on G x Yse defined by {9^ , ) ■ 

Note that the space of differentials forms on G x i^- Yge admits a canonical iden- 
tification with the space of /C-basic differentials forms on G x Yse ■ 

Proposition 2.5. The 2-form 7i'*g(riM„) equal to the K-hasic, G-invariant, 2- 
form ny,^ - d{'S>{l^ , 9^) . 

Proof. Let tti : G x Yse -> ^Ise the map that factorizes TZse- By G-invariance, we 
need only to show that nliVlM,^) equals fiy^^ — d{^Y ,9^) at the point (l,y) G 

GxYse. 

Let iX',v'), {X,v) eQxTy = T(i .y)(G x Yse). We have 
7rl{nMj[{X',v'),{X,v)^ 

= nM{-x'M + v\-XM{y) + v) 

= nMiv',v) + nMix',,i{y),XMiy)) - nM{x',j{y),v)-^nM{XM{y),v') 

= Hy^Av'^v) + (I'Ky), [X',X],)+d{<P^^^,XX{v) - d{^^^^,X,)\y{v') . 

^ V ' " V ' 

A B 

= ny,^ {v', v) - , 9'^) {{X', V% (X, V)) . 

The last equality is due to the fact that A = —{^y {y)^d9^\i){X' ,X) since 
d9^{{Xr, iXy) = ~[X',X],, and B = , ((f ',«'), «))■ 

□ 

2.2. Proper moment map. In this section we study the Hamiltonian actions of a 
real reductive group G on a symplectic manifold (Af, rij\/) that meet the following 
condition: 

CI The action of G on M is proper, 

C2 The moment map <i>^^ : i\/ — > g* is a proper map0. 

''For any compact subset _B C 0* the fiber ^^^{B) is a compact subset of M. 
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The condition C2 imposes that the image of is a closed subset of g*. Let A 
be a compact subset of Image($G), and let A = {^'^^)~^{A) be the corresponding 
compact subset of M. We sec then that, Vg G G 

g-AnA=^lD^ g-AnA=^iD. 

Condition CI tell us then that {56 G \ g ■ A O A ^ (d} is compact, hence {g € 
G \ g ■ AC\ A ^ %} is compact for any compact set A in the image of By taking 
A equal to a point, we get the following 

Lemma 2.6. Under the conditions CI and C2, the image of ^'^j is contained in 
the open subset q*^ = {C £ 0* I G*^ is compact} of strongly elliptic elements. In 
particular, the image of does not contain G g* . 

The previous Lemma gives a strong condition on the reductive Lie group G : it 
may acts in an Hamiltonian fashion on a symplectic manifold, properly and with 
a proper moment map only if q*^ 7^ 0. So G can not be for example SLn{^) for 
n > 3 or a complex reductive Lie group (see Table [1]). 

If we use the last section we see that M = Mge ■ Wc summarize with the following 

Proposition 2.7. • The set Y is a K -invariant symplectic sub-manifold of M , 
with proper moment map $y equal to the restriction of ^'^j to Y. 

• The manifold Gx^Y carries an induced symplectic structure ily — (i(<E>y , 9^). 
The corresponding moment map is [g,y] 1— > g ■ <&y (y). 

• The map n : G x^Y M is a G-equivariant diffeomorphism of Hamiltonian 
G-manifolds. 

• The manifold Y is connected. 

Proof. Thanks to the Cartan decomposition, the third point implies that p x F ~ M 
and then the last point follows. □ 

Let t be the Lie algebra of a maximal torus T in K . Note that g*g 7^ is 
equivalent to the fact that t is a Cartan sub-algcbra of g. Let g*g = g*g n 4* and 
C = n r . We have g,:, - Ad*(G) • = Ad*(G) • i,:,. 

Let A* C i* be the weight lattice : a g A* if ia is the differential of a character 
of T. Let C A* be the set of roots for the action of T on g (K) C. We have 
$H = 5Hc U 5H„ where SHc and are respectively the set of roots for the action of 
T onl®C and p (g) C. We fix a system of positive roots in IHc: let t+ C t* be 
the corresponding Weyl chamber. Let W = W{K, T) be the Weyl group. We have 
then 

C = • (C n t;) = VK-Uet; I (^,qO ^0, Vae$n„} 

= M^-(ClU---UCAr), 

where each Cj is an open cone of the Weyl chamber. 
We recover the following result due to Weinstein [46] . 

Theorem 2.8. • The Kirwan set 1^k{Y) := Image($f ) n t"^ is a closed convex 
locally polyhedral subset contained in one cone Cj. 

• We have Image($f^)/Ad*(G) ~ A^f (F). 



[Q,R]=0 IN THE NON-COMPACT SETTING 



9 



Proof. Since the moment map $y is proper and Y is connected, the Convexity 
Theorem [21 [HI [25l [28] tells us that the Kirwan set Ak{Y) is a closed, convex, 
locally polyhedral, subset of the Weyl chamber. On the other hand, we know that 
the image of ^§ belongs to t*^. Then Ak{Y) C CiU ■ ■ ■ UCn, but since AxiY) is 
convex we have Ak(Y) C Cj for a unique cone Cj. The last point is obvious since 
the isomorphism tt : G Xk Y ^ M satisfies o Tr{[g, y]) = g ■ $y (y). □ 

We finish this section, with the following 

Theorem 2.9. Let {M,ilM,^fj) be an Hamiltonian G-manifold. 

• // the G-action on M is proper, is proper if and only if is proper. 

• Under the conditions CI and C2, we have 

^ Cr(||$G IP) = CrmW) = Cr(||$(^|r) C Y 

Proof. Let us prove the first point. As ||$^/|| > H'&mII '^^'^ implication trivially 
holds. Suppose now that ^'^j is proper. Thanks to Propositions 12.71 and 12.81 we 
know that M = Gx j^Y where K is a A"-Hamiltonian manifold, with proper moment 
map $Y, and with Kirwan set Ak{Y) being a closed set in t*g. Let i? > 0. We 
consider 

. M<n^{meM\ ||<I>f,(m)||2 < i?}, 

• Y<]i = {y eY \ ||$y < R} which is a compact subset of F, 

• /C = Ak{Y) n e t* I ll^lp < R} which is a compact subset of t*^, 

• c(/C) = inf *'2|j|||^ which is strictly positive. 

Wc have to show that M<ii is a compact subset of M. Take m = [fce^,2/], with 
k e K and X ep. Since = ke^ ■ ^yiy), we have 

Hence if m = [ke^,y] £ M<jf, we have y G Y<fl and then ^yiv) = • C for 
some ko G K and G /C. Then we have, for X' = k^^ • X G p, 

IKMIHII [e^'-d, II > ^(e^'-Co,eo) 
L Jr ll^oll 

Thus if m = [ke'^ , y] G A/</j, the vector X is bounded and y belongs to the compact 
subset Y<_R- This proves that Af<fl is compact. 

Let us concentrate to the last point. First we note that since the map ||$^/|P : 
M — >• M is proper, its infimum is reached, and so Cr(||$^j|p) =^ 0. Let — G {G,K}. 
Thanks to ([Hj), we know that 

TO G Cr(||$-^||2) K-{m) ^0<=^ e 0™- 

Since 0,„ C with $^^(m) ^ — ^j, © ^p, we have m G Cr(||<i>^y^|p) only if 
Kpi'?] — 0- Since ^ is strongly elliptic the last condition imposes that — 0. We 
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have proved that Cr(||$|^|p) and Cr(||<i>fj|p) are both contained in {$^^ = 0} = F. 
We have k'' = + and the vector field vanishes on Y. FinaUy we see that 

Cr(||$^f) = Cr(||$^,f) = {yer I [$^)]M(y) = 0} 

= Gii\\<i>^\n 

The last equality is due to the fact that $y is the restriction of to Y. □ 

2.3. Criterion. We have seen in Theorem 12.91 a situation where the properness 
property of the moment maps <I>^^ and $^ are equivalent. In this section, we 
start with a symplectic manifold (Af, f^M) admitting an Hamiltonian action of a 
compact connected Lie group K. We suppose that the moment map is proper. 
Let K' C K he a, closed subgroup. The aim of the section is to give a criterion 
under which the induced moment map $^ is still proper. We start by recalling 
basic facts concerning the notion of asymptotic cone. 

To any non-empty subset C of a real vector space E, we define its asymptotic 
cone As(C) C E as the set formed by the limits y = limfc^oo tkyk where {tk) is a 
sequence of non-negative reals converging to and yk € C. Note that As(C) = {0} 
if and only if C is compact. 

We recall the following basic facts. 

Lemma 2.10. Let Ci, i = 0, 1 be closed and convex subsets of E. 

• We have d + As{Ci) C Q. 

• If CqC] Ci is non-empty we have As(Co) n As(Ci) = As(Co n Ci). 

• If Cq D Ci is non-empty and compact, we have As(Co) H As(Ci) = {0} 

Proof. Let us check the first point. Take z € Ci and y = limfc_>oo tkyk an element 
of As(Ci). Then z -\- y = limfc^oo(l - tk)z -\- tkyk- Since (1 - tk)z -\- tkyk S Ci if 
^fe < 1, we know that z + y ^ Ci since Ci is closed. 

The inclusion As(Co n Ci) C As(Co) n As(Ci) follows from the inclusions Cq n 
Ci C Cj. Let z e Co n Ci and y e As(Co) n As(Ci). Thanks to the first point 
we know that z + C Co n Ci. Then y = hmt^o+ t{z + t^^v) e As(Co n Ci). 

The second point is proved and the last point is a direct consequence of the second 
one. □ 



The following Proposition is a useful tool for finding proper moment map. For 
a closed subgroup K' of K, we denote Trc.t : t* — > (t')* the projection which is the 
dual of the inclusion t' ^ t. The kernel 77^7^(0) is denoted {t')^. 

Proposition 2.11. • Let {M,Qm) be an Hamiltonian K-manifold with a proper 
moment map Let Aif(M) be its Kirwan polyhedron. Let K' (Z K be a closed 

subgroup. Then the following statement are equivalent 

a) the moment map = tt^ t o is proper, 

b) As(AK(M))nX-(«')^ = {0}, 

c) there exists £ > 0, such that the inequality \\ > £\\^m\\ — holds on 
M. 

Proof. If c) does not hold we have a sequence rui S M such that ||$^ (?7ii)j| < 
i||$|S(m,)|| - i, for aU i > 1. Then ||$|S(m,)|| tends to infinity and ]||^ 
tends to zero. We write $^(mi) = ki ■ yi with ki e K and yi e Ak{M). The 
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sequence 7r{/^{(fci • jj"]]') converges to 0. Here we can assume that the sequence 
ki converge to k £ K, and that the sequence converge to y S As{Ak{M)). 
We get then that n{'^i(k ■ y) = 0. In other words, ?/ is a non-zero element in 
As (Ak(M)) n K ■ (r)'^. We have proved b) ^ c). 

The imphcation c) a) is obvious. Let us prove the last implication a) b). 
First we note that the properness of $^ implies that the projection Trc^^ is proper 
when restricted to the closed subset Image($^) — K ■ Ak{M). Let k G K and 
Co e k-AxiM). Then 

k ■ Ak{M) fl Co + («')^ C Image($ff) f| n^^^.int, 

is non-empty and compact. If we apply the last point of Lemma I2.1QI to the closed 
and convex sets k ■ Ak{M) and + (5')^ '^^ get that 

As(fc • Ak{M)) fl As(eo + fi^) = fc • As{Ak{M)) f|(e')^ 

is reduced to {0}. So we have proved that As(Ax(-^'jf)) Pi ^ ' (^')^ = {0} f^'" ^^^Y 
fc e if. □ 

Remark 2.12. When M is a symplectic vector space (EjHe), the moment map 
$11 is quadratic. Then is proper if and only if {^E)~^i^) ~ i^}- 

2.4. Kostant-Souriau line bundle. In the Kostant-Souriau framework, an Hamil- 
tonian G- manifold (M, 51a/, ^m) is pre-quantized if there is an equivariant Hermit- 
ian line bundle Lm with an invariant Hermitian connection V m such that 

(11) C{X)~i{XM)yM ^i{^fi,X) and (Vm)' = -«f^M, 

for every X G g. 

The data {Lm,'^m) is also called a Kostant-Souriau line bundle. Note that 
conditions PT|) imply via the equivariant Bianchi formula the relations (O. 

We suppose now that conditions CI and C2 hold. Then M = G x k Y where 
Y C M is the A'-invariant symplectic sub-manifold defined in Section 12.21 Let 
{Lm,^ m) be a Kostant-Souriau line bundle on M. We denote Ly the restriction 
of the line bundle Lm on Y . The connection Vm induces a isT-invariant connection 
Vy on Ly Y, and we check easily that (_Ly,Vy) is a Kostant-Souriau line 
bundle on Y . 

Reciprocally, if (Ly,Vy) is a Kostant-Souriau line bundle on (Y, fiy, (f>y ), we 
define on M the line bundle Lm ■= {G x Ly)/K equipped with the connection 

Va/ :=Vy + d<^ + z ($1^,61^), 

where d'^ is the de Rhani differential on G. Since 57 a/ = fly — rf($y , S-'^), we 
check easily that (La/,Va/) is a G-equivariant Kostant-Souriau line bundle on 
(Af,f} A/, 

2.5. The case of elliptic orbits. In this section, we consider the examples given 
by the elliptic coadjoint orbits of G, that is i\/ := G • A for some X G t* . The 
Kirillov-Kostant-Souriau symplectic structure Qm is defined by the relation 

nM\miXM\ra,YM\m) = (m, [X,Y]), 

for TO £ M and X,Y e g. The corresponding moment map relatively to the action 
of G on G • A is the inclusion : G • A ^ g*. 

Lemma 2.13. The moment maps <f>^j^ and $^ are proper. 
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Proof. The inclusion ^'^j is proper since the ehiptic orbit AI = G ■ \ is closed in 
0*. If we use the relations ||Af = = - ||$P^||2 and = 

ll-ff/lP + II^Lf, we get ||$|Sf = KIIAIP + The properness of 

then follows. □ 



We work now with an elliptic coadjoint orbit of G, G • A, such that the stabilizer 
subgroup G\ is compact. Then the action of G (and those of any closed subgroup) 
on G • A is proper. 

Let t be the Lie algebra of a maximal torus T in K . Our hypothesis concerning 
the compactness of G\ imposes t to be a Cartan sub- algebra of g. Let $H be the 
set of roots for the action of i on g ® C. We have 91 = 5Hc U where *Hc and 
Din are respectively the set of roots for the action of t on € (g) C and p C. For 
the remaining part of this section, we fix a system of positive roots d\c.+ in ^c- let 
t!j_ C t* be the corresponding Weyl chamber. 

So A is chosen in the Weyl chamber t*^, away from the non-compact wall : 
(a, A) 7^ for all a G Uln- Thanks to Lemma 12.131 wc know that the moment map 
^G A relative to a maximal compact subgroup X C G is proper. The Convexity 
theorem tells us that the set 

Ak(G-A) := Image($^,) n 

is a closed convex locally polyhedral subset of t*. The results of Duflo-Hcckman- 
Vergne [Mj shows that in fact Ak{G-X) is defined by a finite number of inequalities. 
In this paper, we call Ak{G ■ A) the Kirwan polyhedron. 

We consider now a connected reductive subgroup G' C G, such that a Cartan 
involution Q for G leaves G' invariant. Then we have Cartan decompositions g' = 
t' ® p' and = t © p, with f C t and p' C p. Let K C G and K' C G' be the 
corresponding maximal subgroups. 

Consider now the action of G' on (G • A, JIg a). The moment map ^q.x is the 
composition of the inclusion G • A ^ 0* with the orthogonal projection TTg'^g : 0* ^■ 
(0')*. Note that Y :— is non-empty since it contains 7rg/^g(A). 

We are looking to connected reductive subgroups G' C G such that the moment 
map ^Q.x is proper. Theorem 12.91 shows that is equivalent to look at compact sub- 
groups K' C K such that the moment map x is proper. Thanks to Proposition 
12.111 wc have the following criterium 

Proposition 2.14. • The moment map ^q.x is proper if and only if 
As{AK{G-X))nK-{i')^ = {0}. 

We want to stress a property which is peculiar to the reductive Lie groups (in 
comparison with the nilpotent one). 

Proposition 2.15. Let O C g* be a strongly elliptic orbit. Let G' C G be a 
connected reductive subgroup such that $^ = ""o'.g • ^ ^ (fl')* proper. Then for 
any coadjoint orbit O' C (0')* the reduced space 

(12) On7r^,'jO')/G' 

is connected. 

Proof. Since is proper, we have a decomposition O ^ G' x k' Y', where Y' is 
a connected sub-manifold. Then O n nZ^ JO') is empty if O' is not elliptic. And if 
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C = G" • with /i G («')*, we see that O n 7r~,yC)/G' =i ' m)/^ which 

is connected since $y is proper (see the Convexity Theorem [21 [181 [25l [28] ) . ^ 

In general the Kirwan polyhedron A/^(G • A) is not known, but we can use at 
least the following observation. Let 1H„(A) be the set of non-compact roots a such 
that (a, A) > 0. Let us consider the following cone in i* : 

C(A) := ^ M^"a. 
Qe«„(A) 

Lemma 2.16. For the Kirwan polyhedron we have 

Aa-(G- A) c {A + C(A)}, and then As (A/^ (G • A)) c C(A). 

Corollary 2.17. The moment map is proper if C{\) H K ■ [l')^ = {0}. 

Proof. Let C\ be the cone tangent to Ak{G • A) at A : 

GA = R^"-{e-A, AK(G-A)}ct*. 

We have to show that C\ is contained in C(A). Thanks to the result of Sjamaar |40) . 
we know that C\ is determined by a local Hamiltonian model near ii' • A C G • A. 

The maximal torus T of K is still a maximal torus for the stabiliser subgroup Kx : 
let ^ be a Weyl chamber for {Kx, T) which contains t^. Here, we consider the vec- 
tor space p equipped with the linear symplectic structure flx{X, Y) := (A, [X, Y]). 
The group Kx acts in a Hamiltonian fashion on (p, ^Ix). Let us denote by 

AK.(p)ctl,+ 

the corresponding Kirwan polytope (which is a rational cone). Since the stabiliser 
of the point X G M := G • A coincides with the stabiliser subgroup Kx of its image 
by the moment map , the local form of Marie [30] and Guillemin and Sternberg 
[20] tells us that M is symplectomorphic with TiT x p in a neighbourhood of K ■ X. 
Theorem 6.5 of gO] tells us then that Cx = ^kAp)- 

Let us consider the Hamiltonian action of the torus T on (p, ilx)- Let Jx be an 
invariant complex structure on p which is compatible with fix', we can check that 
the weights of the T-action on (p, Ja) are —a, for a G *H„(A). Hence the image 
At(p) of the moment map is equal to the cone generated by the weights a E ?l„(A). 
Finally we have proved that 

Ga = Ak,(p)c At(p)=C(A). 

□ 

3. Quantization commutes with reduction 

Let G be a connected real reductive Lie group and let isT be a maximal connected 
compact subgroup. Let C{, Cg be respectively the center of i and g. In all the section 
we assume that the group G satisfies the following condition 

(13) Z,{c,)=i, 

i.e. the centralizer of C{ in g coincides with t. Hence Cg C C{ C t. 

We make the choice of a maximal torus T in K with Lie algebra t. Note that 
([T3| forces t to be a Cartan sub-algebra of g. Let *H = 9\c U IHn be the set of roots. 
We fix a system of positive roots in D\c- We know also that ([T3| imposes the 
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existence of elements z e C{ such that ad(z) defines a complex structure on p (see 
Section 9 in [SS])- For such element z, we define 

which is invariant relatively to the action of the Weyl group Wk- The union 
9^+ U 9\n{z) defines then a system of positive roots in SH. 

We will be interested to the closed M^^f-invariant cones in t* 

Choi(^) := {C e t* I (/3, > 0, V/3 e 9l„(z)} , 

C{z) := ^ R^V. 

;3elR„(z) 

We recall some basic facts about them. 
Lemma 3.1. We have the following inclusions 

(14) C(z) c Ci,oi(z) c e t* I {Lz)>0}. 

Proof. Since (/3o,/3i) > for any f3k G $H„(z), we see that C{z) C ChoK-z). For 
e e t*, we have (C, z) = -6(|,z) = 2 ^^g^.^^^, (/3, with (/3,0 = (/3,0- Then 
eeChoi(^) implies >0. □ 

Example 3.2. We have the following classical examples: 

G K p 



Sp(n,R) U(n) 52(C") 

S0*(2n) U(n) A^C" 

S0o(2,n) S0(2)xS0(n) C" 

U(p,g) U(p)xU(p) Mp,,(C) 

3.1. Holomorphic coadjoint orbits. The holomorphic coadjoint orbits arc G ■ A 
with A in the interior of Choi(z). These symplectic manifolds possess a G-invariant 
(intcgrable) complex structure J\ which is compatible with the symplectic struc- 
ture 0(3. A (see [3H])- Hence (G • A, fic Ai^A) is a Kahlcr manifold when A € 
Interior(Choi(z)). 

The real .ftT-module p is equipped with the invariant linear symplectic structure 
Vlp{A,B) —h{z,[A,B]). Wc have two families of Hamiltonian A'-manifold : 
K ■ \ y. p and G • A for A e Choi(z). Wc start with the fundamental fact. 

Proposition 3.3. Let A £ Intcrior(Choi(z)). We have 

a) AK(G-A)cA+C(z)cChoi(z), 

b) Ak(G-A) = AK(i^-Axp), 

c) As(Aif(G-A)) = Aa-(p). 

Proof. Point a) is the translation of Lemma [2.161 since the cone C(A) is equal to 
C[z). Point b) is proved in [35]. Another proof is given by Deltour in [T3], by 
showing the stronger result that the Hamiltonian i^-manifolds G • A and K ■ \ x p 
are symplectomorphic. The point c) follows easily from h). □ 

Remark 3.4. When G is one of the groups appearing in Examvle \3.2l the gener- 
ators of the cone Ak{p) can be defined in term of strongly orthogonal roots (see 
Section 5 in [38j ). Note also that Deltour has completely described the facet of the 
polytopes Ak{G ■ A) when Interior(Choi(z)) (see [12] ). 
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Table 2. Involution a such that o{z) — z 



G 


G" 


Sp(n,R) 


Sp(p,R) X Sp(n-p,M) 


Sp(n,R) 


U(p, n-p) 


SO(2,2n) 


U(l,n) 


S0(2,n) 


S0(2,p) X SO(n-p) 


S0*(2n) 


U(p, n - p) 


S0*(2n) 


S0*(2p) X SO*(2n-2p) 


U(n, n) 


Sp(n,M) 


U(n, n) 


S0*(2n) 


U(p,g) 


U(i,j) X U(p-i,g- j) 



Let 5'*(p) be the symmetric algebra of the complex X-module (p,ad(z)): it is 
an admissible representation of K. Let K' be a closed connected subgroup of K. 
We denote by ^^.f^ and by the corresponding moment maps. We have the 
following 

Proposition 3.5. Lei A e Interior (Choi (2)). The following assertions are equivalent 

a) : G • A — > is a proper map, 

b) Ak(p) n K ■ {iY - {0}, 

c) $^ : p — >■ (t')* is a proper map, 

d) {<I>^' = 0} is reduced to {0}, 

e) >S"(p) is an admissible representation of K' . 

Proof. The equivalences a) b) and b) c) follow from Propositions 12.111 and 
13.31 The other equivalences c) <;4> d) <;4> e) are proved in [36] [Section 5] . □ 



Let us consider the moment map $p : p ^ t* . Via the identification t* ~ t, the 
moment map $^ is defined by 

^f{x) = -[x,[z,x]], xep. 

Hence we see that : p — > K is a proper map taking positive values. This 

simple fact and Proposition 13. 51 gives us the following 

Corollary 3.6. Let G' be a connected reductive subgroup ofG, and let A € Interior (Choi 
The moment map is proper when the Lie algebra q' contains Mz. 

Example 3.7. The condition Kz C q' is fulfilled in the following cases: 

(1) G' = SOo(2,p) C G = S0o(2, n) forQ<p< n, 

(2) G' is the identity component of G"^ , where a is an involution of G such that 
a{z) = z (see TableWi), 

(3) G' is the diagonal in G :— G' x • • • x G'. 
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3.2. Holomorphic discrete series. Let A* C t* be the lattice of characters of 
T. We know that the set A*^ := A* n parametrizes the set K of irreducible 
representations of K: for any /i e /\*^_, we denote V^f^ the irreducible representation 
of K with highest weight fi. 

We will be interested in C^^i{z) — 2p„(z) +Choi(^) C Choi(z) where 2pn{z) is the 
sum of the roots of *H„(z). Let us denote 

Theorem 3.8 (Harish-Chandra). For any X G Ghoi(2), there exists a irreducible 
unitary representation of G, denoted , such that the vector space of K-finite 
vectors is 

Here S*{p) is the symmetric algebra of the complex vector space (p,ad(z)). 

3.3. Formal geometric quantization. Let us first recall the definition of the 
geometric quantization of a smooth and compact Hamiltonian manifold. Then we 
show a way of extending the notion of geometric quantization to the case of a 
non-compact Hamiltonian manifold. 

Let be a compact connected Lie group. Let (Af, rijv/,$^) be a Hamiltonian 
fC-manifold which is pre-quantized by the Hermitian line bundle Lm (see Section 

mi). 

Let us recall the notion of geometric quantization when M is compact. Choose 
a ii'-invariant almost complex structure J on M which is compatible with CIm 
in the sense that the symmetric bilinear form r2j\/(-, J-) is a Riemannian metric. 
Let Oli^i be the Dolbeault operator with coefficients in L, and let dj^^^ be its 
(formal) adjoint. The Dolbeault- Dirac operator on M with coefficients in Lm is 
Dlh, = V^{dLM +^Ljj)7 considered as an elliptic operator from Lm) to 

A°'°'^'^iM,LM)- Let R{K) be the representation ring of K. 

Definition 3.9. The geometric quantization of a compact Hamiltonian K -manifold 
(A/, r^Af , <I>^) is the element Qk{M) e R{K) defined as the equivariant index of 
the Dolbeault- Dirac operator D l . 

Let us consider the case of a proper pre-quantized Hamiltonian A'-manifold A/: 
the manifold is (perhaps) non-compact but the moment map $^ : M -> t* is 
supposed to be proper. In this setting, we have two ways of extending the geometric 
quantization procedure. 

First way : Qk°°- ^"^^ defines the formal geometric quantization of M as an 
element Q^°°(Af) that belongs to R-°°{K) := homz{R{K) , Z) [31 [Ml [35] • Let 
us recall the definition. 

For any /i G -ftT which is a regular value of the moment map the reduced space0 
(or symplectic quotient) 

(15) M,:={^fjr\K-^i)/K 

is a compact orbifold equipped with a symplectic structure fl^. Moreover := 
(i|(-$K -j-ij-^-) iSi C-fi)/ Kf^i is a Kostant-Souriau line orbibundle over (Af^jfi^). The 
definition of the index of the Dolbeault-Dirac operator carries over to the orbifold 



The symplectic quotient will bo denoted Af^ ^ when we need more precise notations. 



[Q,R]=0 IN THE NON-COMPACT SETTING 



17 



case, hence Q(Af^) G Z is defined. This notion of geometric quantization extends 
further to the case of singular symplectic quotients [32l[34]. So the integer Q(Af^i) £ 
Z is well defined for every fi € K: in particular Q{M^) = if /i is not in the Kirwan 
polytope AiiiM). 

Definition 3.10. Let {M,ili\j,^fj) be a proper Hamiltonian K-manifold which 
is pre- quantized by a Ko slant- Souriau line bundle L. The formal quantization of 
{M,ilM,^fj) is the element of R^°°{K) defined by 



When M is compact, the fact that 
(16) Qk{M) = Qk^{M) 

is known as the "quantization commutes with reduction" Theorem. This was con- 
jectured by Guillemin-Sternberg in (19j and was first proved by Meinrenken |31| 
and Meinrenken-Sjamaar [32]. Other proofs of (|16p were also given by Tian-Zhang 
[42] and the author [34]. For complete references on the subject the reader should 
consult [mils]. 

One of the main features of the formal geometric quantization Q^°° is summa- 
rized in the following 

Theorem 3.11 ([36]). • Restriction to subgroup. Let M be a pre-quantized 
Hamiltonian K-manifold which is proper. Let H d K be a closed connected Lie 
subgroup such that M is still proper as a Hamiltonian H -manifold. Then Q~^°°{M) 
is H-admissible and we have Q](°°{M)\h = Qh^{M) m R-°°{H). 

• Product. Let M and N be pre-quantized Hamiltonian K -manifolds with M 
is proper and N is compact. Then M x N is a proper pre-quantized Hamiltonian 
K-manifold and we have Q]/"{M x N) ^ Qk^{M) ■ Qk{N) in R-^{K). 



Second way : Q^. When M is a proper pre-quantized Hamiltonian iT-manifold, 
we can define another formal geometric quantization of M through a non-abelian 
localization procedure a la Witten [47] . In [29l |3^ , one proves that an element 

(17) Q%{M) e R-°°{K) 



is well-defined by localizing the index of the Dolbeault-Dirac operator i'Lj,/ on the 
set Cr(||$^|p) of critical points of the square of the moment map. 
The crucial result is that these two procedures coincides [29l [36] . 

Tiieorem 3.12 (Ma-Zhang, Paradan). Let M be a proper pre-quantized Hamilton- 
ian K-manifold. Then, the following equality 

(18) Q7°{M) = Q|(M). 

holds in R-°°{K). 
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3.4. Formal geometric quantization of holomorphic orbits. Let us come 
back to the holomorphic discrete representation V^. Consider a coadjoint orbit 
G ■ X for A S A!j_ in the interior of the chamber Choi (2), so that A is strongly elliptic. 
The action of G on G • A is Hamiltonian, and the line bundle 

L:=Gxk^ Ca 

is a Kostant-Souriau line bundle over G • A ~ G/K\. Here Ca denotes the 1- 
dimcnsional representation of the stabilizer subgroup K\ that can be attached to 
the weight A. 

Thanks to Lemma [2.131 'we know that the moment map <&c.a relatively to the 
action of K on G • A is proper. Hence the reduced spaces 

are compact for any /i G A!j_; and the generalized character S* (G • A) G R~°°{K) 
is well defined. We have proved in [35j [SQ the following 

Theorem 3.13. Let X G A!^ n Interior(Choi(^)). The following equality 

2|(G-A) = V^f ®5'(p) 

holds in R-°"{K). 

This result will be generalized in (PT|) . It shows that Q* (G- A) coincides with the 
vector space of A' -finite vector of the holomorphic discrete representation when 
A G C^^^{z). Note that for A G Intcrior(Choi(z)) \ C^^^{z), the generalized character 
Q* (G • A) can not be associated to an holomorphic discrete representation of G. 

Theorems 13.131 and I3.12[ gives us the following informations concerning the K- 
multiplicities. 

Corollary 3.14. Let A G A^ n Interior(Choi(2)), and yU G A!j.. 

• The multiplicity ofV^ in ® S' {p) is equal to the quantization of the reduced 
space (G • A)^. 

• // [V^f : ® S'{p)] ^ then ^ G A + C(z) C Choi(z)- The last condition 
imposes that > ||A|| or — X. 

Proof. The first point is a consequence of the equality Q*(G • A) = 
Q^°°(G- A). We know then that if [V^ : V^<g)S'{p)] ^ 0, then /i belongs to the Kir- 
wan polytope Ak{G-X). But we know after Lemma [2.161 that Ax(G- A) C A-t-C(z) 
: so /i = A + X];3GiK„(2) with > 0. Finally, we have 

> IIAf, 

and we have = i|A|p only if /i = A. □ 

3.5. Multiplicities of the holomorphic discrete series. We consider now a 
connected reductive subgroup G' C G such that z G g'. Then it is easy to check 
that G' satisfies ((T3)) . Let K' C K he the maximal compact subgroup in G', and let 
T' C T be a maximal torus in K'. Let Choi(z), C^„i(z) C t* and C^^„i(z),C'(J„i(z) C 
(t')* be the corresponding convex cone. For [ G {t, {,3}, we denote tti'^i : [* —J- ([')* 
the canonical projection. We have the following important fact. 
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Proposition 3.15. We have the following relations 

(a) 7ri,,t(Choi(^)) C CU(z), 

(6) 7rr,((A'-Choi(z)) C K'-CLiiz), 

(c) 7r(,,((A--d(z)) C K'-C'Uz), 

(d) ^g',g(G.d(z)) C G'-C'UZ)- 

Proof. Let a G t* be a non-compact root of (g, t). Let C p(8iC the corresponding 
1-dimensional weight space. Then we know that there exists £ ilQa^'Oa] H t such 
that a = —b{ha, •). Note that the half-hne M>°/iq, does not depend of the bihnear 
form b, and that the condition (a, — is equivalent to ha) > for any ^ e t*. 

Let a S t* be a non-compact root of (g, t) such that its restriction a' — 7rt'_t(Q!) 
is a non-compact root of (0',i')- Since the 1-dimensional weight spaces and 
g'^i coincide we have R>"/iq = M>°/iq/ C t'. Then the condition /iq) > is 
equivalent to {t^v ,i{£,) , ha') > 0. Finally we have proved the point (a) : if one has 
{£,, ha) > for any positive non-compact root of (g, i), then (7rt'_t(^), ha') > holds 
for any positive non-compact root of (g',t'). 

Let ^ e Ci,oi{z) and ^' G 7r(-,{(is: • ^) n (t')*. Then g 7rt-,t ° T^tA^ ■ 0- By the 
Convexity theorem [2l [181 IlSl [2l]: we know that ttj {(i^ • is equal to the convex 
hull of Wk£,- But belongs to the Wi<--invariant convex cone Choi(-z), and then 
T^t,tiK ■ C Choi(^)- Finally ^' G 7rt',t(Choi(2)) C C(,oj(z) thanks to the point (a). 

Let ^ G Choi(z)- Since 2p„(2) is iC- invariant, we have K ■ {2pn{z) +£,) = 2pn{z) + 
K ■ ^. Thanks to the point (6), we see that 

7rr,t(i^ • (2p„(z) + C)) = 7rr,t(2p„(2))+7re',t(if-0 

C if'-(^r,«(2p„(z))+CU(z)). 

The if'-invariant term TTc.f (2p„(z)) belongs to (t')* and is equal to 2p'^{z) + Tii' ,i{A) 
where A is the sum of the positive non-compact roots a such that Qa is not included 
in p'(2DC. Hence A G ChoK^) and thanks to point (a) its projection Tii'^A) belongs 
to C^q[(2). The point (c) is then proved. 

Let A G C^fjyz). The coadjoint orbit G • A is contained in g*^, and the moment 
map is proper since z G g' (see Corollarv l3.6p . Then, we know that 

(G • A) = Image(<I>g:j = G' • (^,.,, (G • A) f|(«')*) 

and 

^,',,{G-\){^{tr C ^r,t ° (G • A) 

c • Aif(G- A)) 

C ^t,,((A'.d(z)) [1] 

C K'-C'Uz)- [2] 

Equality [1] is due to the fact that Ak(G- A) C \ + C{z) C C^^y{z) when A G C^^iiz) 
(see Lemma r3.3p . Equality [2] corresponds to c). □ 



Remark 3.16. When the Lie algebra g is simple the set G ■ C\io\{z) C g*e is a 
maximal closed convex G -invariant cone. See [531 Hlj • 
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We finish the section by considering the restriction of the irreducible represen- 
tation Vjp to the reductive subgroup G' . We wih denoted K\io\{z) C K the subset 
f\\(^C^^^{z). We see that K\^o\{z) and GhoK^^) are the same set but they parametrize 
representations of different groups {K and G respectively). 

We start with the 

Proposition 3.17. Let b <E A'hoi(z), A e GhoK^) o,nd ^ G K' . We have 

• If [Vf ■■ V.^'W'] + then M e K,,{z). 

• If {y^' ■■ y^\K\ ^ then M e GUz). 

Proof. Wc use here the " Restriction to subgroup" property of Theorem 13.111 

For the first point, wc know after the Borcl-Wcil Theorem that Vj/^ = Qk{K ■ b), 
and then V^^jA'' = Qk'{K ■ b). Then [V^' : V^^\k'] ^ only if b belongs to 
Image($|^:j = ^^AK ■ b) C K' ■ C"Uz)- But K' ■ C'U{z){i'Y - C'U^) since 
^'hoi(-^) VF/f/ -invariant. We have proved finally that b £ Kl^^i{z). 

For the second point, it works the same. We know that V^\k' = Qk'{G ■ A). 
Hence occurs in the restriction V^\k' only if /i belongs to the image of the 
moment map ^q.^ = T^t'.e ° ^g x- Since Ak{G ■ A) C C^^i{z) (see Lemma [5^ . we 
have 

Image($g.';,) = 7rr,{(lmage($g.;,)) = nf,.,(^K ■ Aif(G • A)) 

where the last inclusion is point (c) of Proposition 13. 15l We have then proved that 
occurs in the restriction V^\k' only if 

M e Imagc($^^.'j n (t')* c (K' ■ C'U^)) n (t')* - C'U^). 

The last equality is due to the fact that C'(|qi(z) is a -invariant subset of (t')*- 

□ 

We denote by G' the unitary dual of G', and by G^ the subset of classes 
of square integrable irreducible unitary representations. The elements of G'^ are 
called discrete series representations of G' and G"^ contains the holomorphic ones : 
GU(^) G',. 

Since the moment map is proper, we know, thanks to the work of T. 

Kobayashi [27] and Duflo- Vargas [15], that the unitary representation is dis- 
cretely admissible relatively to G'. It means that we have an Hilbertian direct 
sum 

V^\g' = mxill) n 
nee;, 

where the multiplicities 771^(11) are finite. In fact, wc can be more precise. 

Proposition 3.18. Let A e Ghoiiz). The multiplicty 771,^(11) is non-zero only if 
n = for some fi € G'^^i{z). This means that we have 

V^\g'^ mAt^)V^?\ 
with m\{^) finite for any fi. 
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Proof. Recall the parametrizatioii of given by Harish-Cliandra. Let (A')* C (t')* 
be the weight lattice. Let ^{q' , t')^ be a choice of positive roots and let p' be half 
the sum of its elements. The set (A')*+p' does not depend of the choice of 9l(g', t')+: 
we denote it by (A')*. Let (t')^ C (t')* be the Weyl chamber corresponding to the 
choice of a set SH(t', t')+ of positive roots. 

The discrete series representations of G' are parametrized by 

G', := e (t')*, b' - regular} n (A'); n (t');. 

At fi € G^, Harish-Chandra associates a square integrable unitary representation 
nj^ : fi is the Harish-Chandra parameter, and 

is the corresponding Blattner parameter. Here p'^ (/j,) is associated to IH^ (/i) . It is 
a classical fact that G (A')* n (t')!J_, and that the representation occurs in 
nj^ I if' with multiplicity one : p,B is the minimal A''-types of 11^ in the sense of 
Vogan. Moreover the map p, n- ps induces a bijection between G^ r\C[^^^{z) and 
GU(^) andwehavenf 

Let p G G'^ such that [H^ : lApIc] 7^ 0. The Proposition will be proved if we 
check that p e CUiz). Since [V^^' : nf |^,] 1, we have [V^^J : V^\k'] ^ 0. 
Thanks to Proposition 13.171 we have then pb ^ ^'hoi('^) • I^b ~ ^^Pni^) + ? with 
C e CU(^). Hence 

/i = (p;(z) + + - p'M) + e. 

The term p' := Pn(-z) + p'c is associated to the choice of positive roots 9^(0', t')+ := 
t')+ U Thus we have (p', a) > for any a € m'„{z). 

The term ^'^(z) — Pnip) is equal to the sum 

E « 

(Q,/i)<0 

and then (p'j(z) — p'j(p), a) > for any a G D\'^{z). Finally we have proved that 
{p, a) = {p'^{z) + p'^, a) + (p'niz) - p'M,a) + a) 

>0 >0 >0 

is positive for any a E 5H^(z), thus p G C^q[(z). □ 

3.6. Jakobsen- Vergne's formula. The aim of this section is to give a direct proof 
of the following result of Jakobsen- Vergne . 

Theorem 3.19 (Jakobsen- Vergne). The multiplicity m\{p) is equal to the multi- 
plicity of the representation ofV^ in S*{p/p') ® V^\k'- 

Let us denote 
(19) R-°°{G,z) 

the Z-module formed by the infinite sum X^agGi liz) ^\ with m.\ € Z. Similarly, 
we define R~°°{K, z) C R~°°{K) as the sub- module formed by the infinite sum 
S/^eA* '^M where € Z is non-zero only if /i e K\io\{z). 
We have the following basic result. 
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Lemma 3.20. • The restriction to K defines a morphism 
(20) rK,G-R'"°iG,z)^R-'^{K,z) 
that is injective. 

• The product by S"(p) defines a map from R^°°{K,z) into itself. 

Proof. Let us prove the first point. Thanks to Corollary 13.141 we have 
E^sa; Q{{G ■ \),) V^. Then 

vk,g{ J2 '^^^^) E "^aV^Ik 

AGGhoi(^) A(EGhoi(z) 



K 



We know that Q{{G ■ X)^) ^ only if ||A|| < and ^ G C{z). Hence the sum 
:= X^AeGh i(z) ^>^Q((G ■ A)^) has a finite number of non-zero term and ^ 
only if e Choi(z). 

Let A = X^AeGh 1(2)"^-^ ^a^ ^ non-zero element in R^°° {G, z). Let Aa G 
GhoK^^) such that ||Aa|| is minimal among the set {||A|j | m\ ^ 0}. Let rK,G{A) = 
E^n^i;^. Then 

nx^ := + ^ mxQ{{G ■ X)x^). 

But mx = if ||A|| < ||Aa|1 and Q{{G ■ \)xa) = if A 7^ Aa and i|A|| > \\\a\\ (see 
second point of CoroUarv I3.14p . We have checked that nxj^ = toa^ 7^ and then 
vk,g{A) ^ 0. 



Let us check the second point. Let A = X^^ga* ^ ^ °°{K,z). Then 

= E( E n,Q{{G-„)e))vi'. 



p,ec{z) 

Like before, the term Q{{G ■ ^)g) is non-zero only if ||yLt|| < \\9\\ and 9 € C{z). Hence 
the sum X);jgc(2) ''^p.QiiG- fi)e) has a finite number of non-zero term and is non-zero 
only if 6'eC(z). □ 

Let us consider the similar morphism yk' ,G' ■ R~°°{G' , z) R~°°{K' , z) for the 
reductive subgroup G' . We consider the following elements of R~°°{G', z): 

V^\g' = "^A(M)n?'' and 

S := J2 Mt^)y^'^ 

where nx{ti) := SVf : S"(p/p') ®V^\k']- Theorem will be proved if wc check 
that rK',G'iV^\G') - rK,,G'iS). But ta-.g^^a'^Ig') = V^\k' = 5'(p) ® V^^\k', 
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and 

M6G^„i(^) 

The second point of Lemma [3.201 insures that the product in [1] is well-defined. 
We need to explain the last equality. Note that [V^' , S*{p/p')(^V^\k'] ^ implies 

[V;f' , S'{p)(g)V^\K'] + and then ^ G G'^J^z) (see Proposition[S371) . This insures 
that the sum 

is equal to S"(p/p') «) V^\k'- 

3.7. Formal geometric quantization of G-actions. In this section we consider 
the Hamiltonian action of a connected real reductive Lie group G on a symplectic 
manifold {M, SI a/). We suppose that the action of G on M is proper and that the 
moment map <I>^^ : AI — > q* is proper. We know that we have a global slice Y C M 
such that 

M-GxkY, 

and that the G-orbits in the image of <I>^^ are parametrized by the Kirwan polytope 

Let us suppose the existence of a G-equivariant pre-quantum line bundle Lm — > 
M. Note that Lm is completely determined by its restriction Ly — >■ 1^ to the sub- 
manifold Y: here Ly is a X-equivariant pre-quantum line bundle over (Y, Oy). For 
any dominant weight /i, we see that the reduce space 

coincides with Y^^k := {^^j)^^(K ■ ij)/K. Hence its quantization 

Q{.M^,g) Q{Yf,.K) e Z 

is well-defined (see Section I575|) . 

Wc suppose also that G satisfies and we fix a complex structure ad(z) on 
p. Let Cy^ay{z) C i* be the corresponding cone. 

Lemma 3.21. Let Let {M, flm, ^m) be a Hamiltonian manifold. Suppose that the 
image of is contained in G ■ Cj'^qi(2) C 0*e. Then : 

(1) the Kirwan polytopes IS.k{Y) C 1^k{M) are contained in C^oi(-z); 

(2) the functions ($y,z) and {^fj,z) take strictly positive values. 

Proof. The Kirwan polytope Ak{M) = iTt^g (lmage(<f>^y^)) n t^J. is contained in 

U Ak(G.A)cC(z), 
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where the last inclusion is a consequence of point (a) in Proposition 13.31 The first 
point is proved. Hence we get, thanks to ([T4|) . the following relations 

Image(($^,z)) C Iniage(($f„ C (Aa'(A/),z) C (d(^),^> C [c,+c^[ 

with c = z) = i dimp. □ 

We have the following notion of formal geometric quantization that extends the 
case of compact Lie group actions. 

Definition 3.22. Let (M, Q.m, $a/) be a pre- quantized Hamiltonian manifold, such 
that the moment map is a proper map from M into G ■C^^^y{z). Then we define 
the formal geometric quantization of M as the following element of R~°°{G, z): 

Let tk.g ■ R~°°{G, z) R~°°{K, z) be the restriction morphism defined in 
Lemma 13.201 Recall that in the setting of Definition I3.22[ the moment map is 
proper (see Theorem l2.9p . Then the formal geometric quantization of M relatively 
to the iC-action is well-defined : Q]^°°{M) € R-°°{K). 

We have proved in Theorem 12. 9[ that the sets of critical points of the function 
II^mIP' II^mP ll^y P 3,re equal. We will be interested by one of the following 
hypothesis: 

Assumption 3.23. • Al The set CT{\\^fj\\^) is compact. 

• A2 The map z) : M is proper. 

In the following Lemma, we exhibit examples where the Assumptions Al or A2 
are satisfied. 

Lemma 3.24. • Suppose that we are in the algebraic setting: the manifold M 
is real algebraic and the map is a proper algebraic map. Then Cr(||$^^||^) is 
compact. 

• Suppose that the Lie algebra g is simple. Then, in the context of Definition 
\3.2Si the map z) : M ^M. is proper. 

Proof. Let us prove the first point. The map ip := : M — > M is a real 

algebraic map on a real algebraic manifold. Thus the set Cr((p) is an algebraic 
variety, and by a standard Theorem of Whitney, it as a finite number of connected 
components Ci, • • • , Cp. Each Cj is contained in (p^^((^(Ci)) which is compact since 
If is proper. The proof is completed. 

For the second point we use the result of Proposition 12.111 and the facts that, 
since Q is simple, [p,p] = I and the center C{ of I is reduced to Mz. 

The function which is the moment map for the 5^-action, is proper 

if and only if As(Ak(M)) n (Kz)-^ = {0}. Since Ak(M) C C^oi(z) (see Lemma 
IMD), it is sufficient to prove that C\,o\{z) n (Mz)^ = {0}. Let ^ g C\,o\{z)- We have 
(C,z) = -6(1, z) = 2E^gg,^(^.)(/3,0 with (/3,0 = (/3,C) > 0. If (C,z) = 0, we must 
have {PtC) = 0,V/3 e 5H„(z) or equivalently [^,p] = 0. Then £^ commutes with all 
elements in [p,p] — t, i.e. ^ G C( = Rz. Finally, we have proved that ^ S (Rz)-'- and 
f e Mz, hence ^ = 0. □ 



[Q,R]=0 IN THE NON-COMPACT SETTING 



25 



We can now state the main result of this section. 

Theorem 3.25. // Assumptions Al or A2 are satisfied, we have the following 
relation 

vk.,g{Qg°^{M))^Q'^^{M). 

Proof. We have 

= ( Y. QiY,)y^)®s'{p) [1] 

A«6A;nc£„i(^) 
= Qir{Y)®S'{v) [2]. 
Note that the product in [1] and [2] are well defined thanks to Lemma [3.201 In 
[2] we use the fact that Q^°°(F) = X^^eAi^nC (z) Qi^iJ-W^^ since by hypothesis 
/S.k{Y) C C(^jj[(z). So Theorem 13.251 follows from the following equality 
(21) Q-^--{GxkY) = Q-^^{Y)®S'{p), 

that will be proved in Sections 14.41 and 14.61 □ 



We consider now a connected reductive subgroup G' C G such that z e g'. The 
coadjoint orbit G • A is pre-quantized when A G Ghoi(z) and we have obviously 
Qq°°(G • A) = . The moment map <^^'.^ : G • A ^ (g')* relative to the G'-action 
on G • A is proper. In fact we have more : the map {^q.xi z) : G • A — > M is proper, 
thus Assumption A2 holds. 

We are interested in the compact reduced spaces 

(G.A)^,G' i^'s'.,r\G'■^,)/G', 

for ^' G Ghoi(z). We are now able to prove the following 

Theorem 3.26. Let A € Ghoi(z)- Then we have the following relation 

V^\g'^Qg^{G-X) 

in R~°°{G' , z). It means that for any fi G G'-^^y{z), the multiplicity of the represen- 
tation in the restriction V^\g' is equal to the geometric quantization 



q((g-a), 



e Z 



of the (compact) reduced space (G • X)^^g' ■ 

Proof. Since the restriction morphism yk',G' ■ R~°°{G' ,z) — > R^°°{K' , z) is injec- 
tive (see Lemma 13201) it suffices to prove that 

(22) TK'^G' {v^\g') - rK'^G' (Qg'°°(G ■ ^)) • 

But the left hand side of ([^^ is equal to the restriction V^\k', while the right 
hand side is equal to Q^?° (G • A) thanks to Theorem 13.251 Theorem 13.131 tells us 
that QJ^°°{G ■ A) = and the functoriality of the quantization process 

(see Theorem 13. lip insures that the restriction V^\k' = QJ^°°{G ■ X)\k' is equal to 
S^?°(G-A). □ 
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We can finish this section by extending the functoriality of the quantization 
process Q^°° relatively to the restrictions. 

Definition 3.27. An element m := X^AGGh ^ R^°°(G, z) is admissible 

relatively to G' if the projection TTg/.j, is proper when restricted to the subset G ■ 
Support(m) C G ■ C^oi(z), where Support(TO) = {A | m> 7^ 0} C Ghoi(-z)- 
The same definition holds for the couple {K',K). 

When m e R^°°{G, z) is G'-admissible, we can define its restriction 

TCG',G{m) := ^ rnxV^lc 

AGGhoi(2) 

= E ( E ^^xQiiG-X),,G')\v^' eR-^{G',z). 

Note that for any fj. e G'^^i{z) the sum X^AeGh "^aQ((G- A)^,g') has only a finite 
number of non-zero term. Similarly, when n = J^beR, i{z) Vj^ G R^°°{K, z) is 
i^'-admissible, we can define its restriction 

rK',K{n) := ^ rn,V^\K' 
f)e^hoi(z) 

E ( E nbQ{{K-b)a,K')\vf ^R-°°{K',z). 

We will used the following Lemma that will be proved in the Appendix. 

Lemma 3.28. Let me R^°°{G,z) that is G' -admissible. ThenYK,G{m) e R^°°{K,z) 
is K' -admissible and the following relation 

yk'^k o rK,Gim) = rK',G' ° i"G',g(™) 

holds in R-°^{K',z). 

We finish this section with the following 

Theorem 3.29. Let {M,nM,^%i) be a pre-quantized Hamiltonian manifold. Sup- 
pose that Imagc($^j) C G ■ C\io\{z), and that the map ($^/, z) is proper. Let G' be 
a reductive subgroup such that z € q' . Then: 

• The map ^fj is proper and Image(<I>f^) C G' • C'fjQj(z). So Q^?°{M) g 
R^°°{G' , z) is well defined. 

• The element Qq°°{M) € R^°°{G, z) is G'-admissible and we have 

VG',G{QG°°{M))=Q-ar{M). 

Proof. The map is proper since is proper. The point concerning the 

image of is a consequence of point (d) in Proposition l3.15l Let m = Qq°°{M) £ 
R~°°{G, z). By definition Support(TO) is contained in Image($^), and then G ■ 
Support(m) C Image($^j). Since the moment map is proper, we know that 
the projection TTg/ g is proper when restricted to Image($^j). This implies that 
m £ R^°"{G,z) is G'-admissible. 
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We have then 

rK',G'OrG',G{QG°°iM)) = rK,,KOrK^G{QG°°iM)) [1] 

= rK',K{Q]riM)) [2] 
= QK^iM). [3] 

Here [1] fohows from Lemma I3.28[ [2] fohows from Theorem 13.251 and [3] is the 
consequence of Theorem 13.111 We have checked that 

rK',G' o VG.,G {Qg°°{M)) = Qk^{M) = rK'^G' {Qg^{M)) . 

Since the map Tj^' G' is injective, it fohows that Tg'^g {Qg°°{M)) — Q^^{M). □ 

3.8. Geometric quantization of the slice Y. Let A £ G[^^^{z). Considei0 the 
coadjoint orbit G" • A associated to the holomorphic discrete series representation 
. Let G be a reductive subgroup of G' such that z S g. We know that we have 
a geometric decomposition 

G' -X^G-xkY 

where y C G' • A is a closed i^-invariant symplectic sub-manifold. 

We have two ways of computing the multiplicity of m\{fi) of in . First, 
after Jakobsen-Vergne, we know that 

mxi^l)^ [yf :5*(p7p)®VfV], 

and Theorem 13.261 tells us also that 

nl^{^l) = Q((G' • \)^,g) = Q{Y^,k). 

We would like to understand a priori why Q(F^,if) = [V^^ : S"(p'/p) ^ Vx^'\k] for 
any fi G K\^oi{z), or equivalently why we have the relation 

(23) Q],'-{Y)^S'{p'/p)^vf\K. 

Note that Assumption A2 holds in this setting : the map is proper. 

Let us consider a more general situation. Let (Af, ^ prc-quantized 

Hamiltonian G-manifold. We suppose the G-action proper, and that the moment 
map $^ takes values in G • C{Jq[(z). We suppose furthermore that Assumption A2 
holds. Let F C M be the symplectic slice. The aim of this section is to compute 
Q^°°(r) in a way similar to 

Let A" be a connected component of Y^ . Let us fix a JT-invariant almost complex 
structure on X which is compatible with the symplectic structure. Let 

be the corresponding Riemann-Roch character (see Section \4:.2\i . Recall that, if 
Lx denotes the restriction of the Kostant-Souriau line bundle Lm on A', we have 
QKiX)=RR''iX,Lx). 

Let Afx X he the normal bundle of A" in F : it inherits a complex structure 
Jx and a linear endomorphism £{z) on the fibres. We have a decomposition Mx — 



^In this section, we interchange the role of the groups G and G' in order to minimize the primes 
in the notation. 
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J2a£R-^x where Af^ = {v G Afx I = aJx{v)} is a sub-bundle of A/'a'. We 
define the vector bundle A/'_y'^ :~ ^±a>o^x and 

Theorem 3.30. We have the following equality in R^°°{K): 

X 

where rx is the complex rank of J\f^'^ . 
The proof will be given in Seetion 14.51 

Let us explain how the formulas of Jakobscn-Vergnc ean be recover with Theorem 
I330l When M = G" • A, the sub-manifolds and Af^ are both equal to K' ■ X. 
The restriction of the Kostant-Souriau line bundle Lm M on is [Ca] := 
K' Xk'^Cx K' ■ A. Relation ^ tells us that the normal bundle Afi of F in M is 
equal to the trivial bundle p x Y, and the normal bundle J\f2 of Y^ in M is equal 
to Y^ X p'. Hence the normal bundle of Y^- in Y is 

N = N2/{Ni\y') = Y' X (p7p). 

We check that N''^'^ = : this is due to the fact that the function (^g/.^i z) takes 
its minimal value on Y^ = K' ■ X (see Lemma 7.3 in [S^). So \J\f\^ = A/" is the 
trivial complex bundle with fiber (p'/p, ad(z)). Theorem 13 . 301 gives 

Q^TiY) = RR^(/v'-A,[Ca]®5'(p7p)) 

= RR^'(i^'. A,[Ca])|k®5'(p7p) 
= vf\K^S'{p'/p). [1] 

In [1], we use that RR^'(ii" • A, [Ca]) = V^' thanks to the Borel-Wcil theorem. 

4. Transversally elliptic operators 

The aim of this section is to give a proof of Theorem 13.251 and 13.301 In the first 
section, we briefly introduce the material we need from the theory of transversally 
elliptic operator. And in Section [4.31 we recall the definition of the geometric quan- 
tization process Q*. In the rest of this paper, K will denoted a connected compact 
Lie group. 

4.1. Transversally elliptic operators. Here we give the basic definitions from 
the theory of transversally elliptic symbols (or operators) defined by Atiyah-Singer 
in [T] . For an axiomatic treatment of the index morphism see Berline-Vergne [H H] 
and Paradan-Vergne [37]. For a short introduction see [34] . 

Let X he a compact /sT-manifold. Let p : TX X he the projection, and let 
(— , —)x be a i^-invariant Riemannian metric. If are isT-equi variant complex 

vector bundles over A", a X-equivariant morphism 

cr e T{TX,\iotcl{p*E^,p*E^)) 

is called a symbol on X. The subset of all (x, v) e TX wherc0 a{x^ v) : E^ ^ E^ is 
not invertible is called the characteristic set of cr, and is denoted by Char((T). 



^The map a{x,v) will be also denote (7\x{v) 
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In the following, the product of a symbol cr by a complex vector bundle F — > M, 
is the symbol a<SiF defined by <7<^F{x,v) = a{x,v)®ldF^ from E°^®Fx to E^^Fx- 
Note that Char(cr ® F) = Char(cr). 

Let TkX be the following subset of TX : 

TkX = G TA", (w,XA'(a;))^ for all X e «} . 

A symbol cr is elliptic if cr is invertible outside a compact subset of TX (i.e. 
Char(cr) is compact), and is K-transversally elliptic if the restriction of cr to Tk'^ 
is invertible outside a compact subset of TkX (i.e. Char(cr) n TrX is compact). 
An elliptic symbol cr defines an element in the equivariant K'^-theory of TX with 
compact support, which is denoted by 'KP^^iTX), and the index of cr is a virtual finite 
dimensional representation of K, that we denote Index;^ (cr) g R{K) [SHHEKS]. 

A K-transversally elliptic symbol a defines an element of K^(Ti<-A'), and the 
index of cr is defined as a trace class virtual representation of K , that we still denote 
Indexf (cr) eR~°°{K) [1]. 

Using the excision property, one can easily show that the index map Index^ : 
K^(TxZ^) — > R''°°{K) is still defined when is a X-invariant relatively compact 
open subset of a X-nianifold (see [5J [section 3.1]). 

Suppose now that the group K is equal to the product Ki x K2 ■ An intermediate 
notion between the "ellipticity" and "Xi x /ir2-transversal ellipticity" is the "i^i- 
transvcrsal ellipticity" . When a Ki x 7ir2-equivariant symbol cr is Ki -transversally 
elliptic, its index Indcx^^ '^'^^ (cr) G R~°°[Ki x if 2), viewed as a generalized function 
on A'l X K2 , is smooth relatively to the variable in K2 [2 [HI [37] . It implies that : 

. Indcx^i><^^(fT) = J2^^^^ ex ® V^' with Ox e RiK2), 

• we can restrict Index^^ ^ (cr) to the subgroup Ki and 

(24) Index^^^^^(a)|K, = J2 dim{0x)V^''' = Indcx^^ (a). 

Here dim : R{K2) ^ Z is the morphism induced by the restriction to 1 G K2- 

Let us recall the multiplicative property of the index map for the product of 
manifolds that was proved by Atiyah-Singer in [T]. Consider a compact Lie group 
K2 acting on two manifolds Xi and X2, and assume that another compact Lie 
group Ki acts on Xi commuting with the action of K2. The external product of 
complexes on TXi and TA'2 induces a multiplication (see [1]): 

ixKsCl'ifi'^l) X K5f2(T_R-2A'2) > K.^ji^y^ji^{T KixK2{^l X ^2))- 

Let us recall the definition of this external product. For k = 1,2, we consider 
equivariant morphism^ crj, : E'^ — > 5^ on TXk- We consider the equivariant 
morphism on T{Xi x X2) 

ai Q <T2 : St ^2 ® ^1 ® £2 — ^ £1 ®£t ® £t ® £2 

defined by 

/ ai®Id -Id® (72* \ 



In order to simplify the notation, we do not make the distinctions between vector bundles on 
TX and on X. 
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We see that the set Char((Ti0(T2) C TXi x TX2 is equal to Char(CTi) x Char(cr2)- 
We suppose now that the morphisms ak are respectively i^fc-transversally elliptic. 
Since TkixK2 {'^1 x -^2) 7^ Tri^i x Tk2'^2, the morphism cti 0(72 is not necessarily 
ilTi X i4r2-transversally elliptic. Nevertheless, if cr2 is taken almost homogeneous^ then 
the morphism ai (T2 is A'l x A"2-transversally elliptic (see [37]). So the exterior 
product fli 02 is the k-theory class defined by cti (72, where = [a^] and 172 is 
taken almost homogeneous. 

The following property is a useful tool (see [T] [Lecture 3] and [37] )• 

Theorem 4.1 (Multiplicative property). For any [ai] e K^^-^ ^^^^ (Ti^-jAfi) and any 
[(^2] e K%^{Tk2X2) we have 

Inde^^l^g^ila,] [a^]) = Indcxf^^ ([^i]) Index^^^ ([,72]). 

4.2. Riemann-Roch character. Let M be a compact /\-manifold equipped with 
an invariant almost complex structure J. Let p : TM — >■ M be the projection. 
The complex vector bundle {T* M)'^'^ is AT-equivariantly identified with the tan- 
gent bundle TM equipped with the complex structure J. Let h be the Hcrmitian 
structure on (TM, J) defined by : h{v,w) ~ il{v,Jw) — iil{v,w) for v,w E TM. 
The symbol 

Thom(A/, J) e r (TM,hom(p*(Ag''""TM), p* {A°c'^''T M))) 
at (m, v) G TM is equal to the Clifford map 

(26) c,n{v) : A^'^™T„,M — > A^'^'^T^M, 

where Cmiv)^ = vAw-l{v)w for w G AJT^M. Here l{v) : A* T,„7\f A*~"^TmM 
denotes the contraction map relative to h. Since Cm(w)^ = — ||f |pld, the map €,„(«) 
is invertible for all v 0. Hence the characteristic set of Thom(M, J) corresponds 
to the 0-scction of TM. 

Definition 4.2. To any K-equivariant complex vector bundle E — > M , we associate 
its Riemann-Roch character 

RR^(Af, A) := IndexA/(Thom(Af, J) ® E) e R{K). 

Remark 4.3. The character RR^(A/, _E) is equal to the equivariant index of the 
Dolbeault-Dirac operator T>e '■= \/2{dE + d^), since Thom(M, J) ® E corresponds 
to the principal symbol ofT>E (see ^[Proposition 3.67]). 

4.3. Definition of Q*. Let (Af, iljv/,$^) be a compact Hamiltonian iiT-manifold 
pre-quantized by an equivariant line bundle Lm- Let J be an invariant almost com- 
plex structure compatible with fl. Let RR^(A/, — ) be the corresponding Riemann- 
Roch character. The topological index of Thom(M, J) Lm € Kj^(TAf) is equal 
to the analytical index of the Dolbeault-Dirac operator \/2{dLM + ^Lm) ■ 

(27) Qk(M) =RR^(Af,LM). 

When M is not compact the topological index of Thom(M, J) Lm is not 
defined. In order to extend the notion of geometric quantization to this setting we 
deform the symbol Thom(M, J) L in the "Witten" way [31 [351 [13] ■ Consider 
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the identification ^ i-> ^,fi* — > fi defined by a X-invariant scalar product on i*. We 
define the Kirwan vector field on M : 

(28) K„ = (^$f^(m)^ (m), m G M. 

Definition 4.4. T/ie symbol Tliom(Af , J) ® L pushed by the vector field k is the 
symbol c" defined by the relation 

c'^lmiv) = Thoni(M, J) ® LU(w - k„) 

/or any {m,v) € TAf. More generally, if E ^ M is an equivariant complex vector 
bundle, one defines the symbol with the same relation (with E at the place of 
L). 

Note that d^\m{v) is invertible except if w = k,„. If furthermore v belongs to 
the subset Tj^M of tangent vectors orthogonal to the A'-orbits, then v = and 
K„i = 0. Indeed Km is tangent to ii" • m while v is orthogonal. 

Since k is the Hamiltonian vector field of the function the set of zeros 

of K coincides with the set of critical points of Finally we have 

Char(c'^) n TkM ~ Cii\\<^fj\\^). 

In general Cr(||$^|p) is not compact, so c** does not define a transversally elliptic 
symbol on AI. In order to define a kind of index of c**, we proceed as follows. For 
any invariant open relatively compact subset U C M the set Char(c''|[/) n TkU ~ 
Cr(||$|p) n ?7 is compact when 

(29) (9C/nCr(||$|l2) ==0. 
When holds wc denote 

(30) Q|(C/):=Index^(c'^|t,) G Kr{K) 

the equivariant index of the transversally elliptic symbol c'^\u- 

Let us recall the description of the critical points of ||<i?^|P, when the moment 
map is proper. We knows that m G Cr(||$^^j|p) if and only if /^Mi'm) — for 
(3 = <I>(to). Hence the set Cr(||$^|p) has the following decomposition 

Cr(|i<i>f,f ) = U Af^n($^,)-i(/?)= U K-iM~^n{'i>f,r\(3)) , 
pet' peB ' 5" ' 

where B is a subset of the Weyl chamber t!]_. We denote by B^. C t* the open ball 
G t* I ll^ll < r}. The following Proposition is proved in [39] . 

Proposition 4.5. • For any r > 0, the set BD Br is finite. 

• The set of singular values of : M — >■ R forms a sequence < ri < r2 < 

. . . < rk < ■ ■ ■ which is finite if and only if Cr(||$;^|p) is compact. In the other 
case limfe^oo ^^=00. 

For any /3 G we consider a relatively compact open invariant neighbourhood 
Up of Zj3 such that Cr(|l$f^||^) fMAp = Zp. The excision property teU us that the 
generalized character QxiUp) — Index^^ (c'' j^^^ ) does not depend of the choice of 
Up. In order to simplify the notations we consider the following 
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Definition 4.6. • We denote Q^{M) G R^^°°{K) the equivariant inde^ of the 
transversally elliptic symbol c'^\u^. 

• When E ^ M is an equivariant complex vector bundle, we denote RR^(Af, i?) 
the equivariant index of the transversally elliptic symbol c'^\uf3- 

The following crucial property is proved in [551 ISH] ■ 

Theorem 4.7. A representation occurs in the generalized character Q^{M) g 
R-^{K) only tf \\X\\ > 

Definition 4.8. The generalized character Q%{M) E R^°°{K) is defined by 

(31) Q%{M) = J2 QkW. 

The sum ([31]) converges in R~°°{K) since we know after Theorem 14.71 that the 
multiplicity of in Q^(M) is zero when ||/3||>||A||. 

We finish this section, by recalling a result that will be needed in Section H75] 
Suppose that t = ti ® ^2 where [4i,{2] = and ti arc the Lie algebras of closed 
connected subgroups Ki. We assume that the moment map : M — >■ J J relative 
to the ifi-action is proper. Let us explain how we can use the iT-invariant proper 
map II^M IP instead of in order to defined the the geometric quantization 

Ql{M). 

Let us choose t = ti ® t2 such that ti C is a maximal abclian sub-algcbras. We 
start a decomposition 

(32) Cr(|KMP)= U 5-(A^^n($f/)-^(/3)) , 
with Si C ti- 

Let Ki be the Hamiltonian vector field of ^H^^^lp, and let c^^ be the corre- 
sponding pushed symbol. For any /? G Si, we consider a relatively compact open 
if-invariant neig hbourhood U} of Zj such that Cr(||$f/ f)C]Ul = Z^. We denote 

Q^^{M) £ R~°°{K) the equivariant index of the iiTi -transversally elliptic symbol 
c'^^Ij^i. Theorem 14.71 admits the following extension 

Theorem 4.9. A representation occurs in the generalized character Q^^{M) 
only if ||Ai|| > ||/?||. Here A G A* C t* is decomposed m A = Ai © A2 with Xi G i*. 

Like in Definition l4.8[ we can define the generalized character QJ-^ (M) G R^°°{K) 

by 

(33) Q*^(Af)= ^ Q^i(Af). 

In |39] [Section 4.1], we prove the following 

Theorem 4.10. Let (M,r2M,$M) be a proper Hamiltonian K -manifold that is 
pre- quantized. If the moment map : AI — > tl is proper, we have 

in R-'^iK). 

^The index of c'^Ij^^ was denoted RR^ (M, L) in [Sl] 
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4.4. Proof of Theorem 13.251 under Assumption Al. In this section we con- 
sider the manifold M ~ G Xk Y, where (Y, fly, $y ) is a Hamiltonian if-manifold 
pre-quantized by a line bundle Ly- We suppose that the moment map $y is proper, 
and that the Kirwan polytope Ak(Y) is contained in the cone C^^i{z) C t*g. 

Then on M, we have an induced G-invariant symplectic form ^Im and a moment 
map : M ^ Q* defined by ^fj{[g,y]) = g ■ ^fjiy). We know that the line 
bundle Lm = (G x Ly)/ K pre-quantizes the Hamiltonian manifold (M, f2j\f , $^/). 
Let us consider the X-action on M: the moment map $^ is also proper. 

We are then in a setting where the formal geometric quantization of M and Y 
relatively to the i^-action are weh defined: Q%{M), Q%{Y) € R-^{K). The aim 
of this section is to prove that 

(34) Q%{M)^Q%{Y)®S'{v), 

when (Af, 51^/, '^'m) satisfies Assumption Al. Then the set (see Theorem 12. 9p 
Cr(||$^f ) = Cr(||$^,f ) = Cr(||$^'f ) ^ |J . {yf^ n {^§y\P)) 

is compact: the parametrizing set B is finite. So we have Q%{M) = J^peB QkI^'^) 
and Q* (F) = J2i3eB QkO^)^ reduce to show the following 

Theorem 4.11. For any f3 Cz B, the following relation 

(35) Q^(A/) = Q^(y)®5-(p), 
holds in R-°°{K). 

Proof. Let km be the Kirwan vector field on M associated to the moment map 
^fj. Let Jm a ii'-invariant almost complex structure compatible with Hm, and let 
Up C M he a (small) neighbourhood of Zp in M. 

The symbol Thom(A/, Jm) ® Lm pushed by the vector field k,m is denoted c^^. 
By definition Q^(A/) is the equivariant index of the A'-transversally elliptic symbol 
^mIw/s- Note that Q^{M) does not depend on the choice of the neighbourhood Ujs 
nor on the choice of the almost complex structure on W/j C M. 

We use the if-diffeomorphism : p x F ~ M defined by (p{X, y) = [e-^ , y]. The 
Kirwan vector field KpxY '■= j\f) is defined by the relations : Kpxy{X,y) = 
{KiiX,y),K2iX,y)) e Tp X r whercS 

Ki{X,y) = Ayiy), K2{X,y)^yA,X] and A = [e^ • i|%)](. 

The Kostant-Souriau line bundle lp*{Lm) is A'-diffeomorphic with Ly since Y 
is a deformation retract of p x F. Let us compute the pull-back of the symplectic 
form ^IpxY ~ ip*{Vt]^i) at (0,?/). For v,v' G TyY and r/ e Top = p, we have 

%xY{ri®v,7^'®v') = ilMiv ®ri ■ y,v' (Br]' ■ y) 

= ny{v,v') + {^^{y),[rj,r/]). 

Lemma 4.12. [r/, ad(z)r/]) = — ([C, [ZtTi]) < for any G K ■C^^i{z) and any 

vep\{0}. 



[Z\f and [X]p are respectively the t and p components of Z G g. 
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Proof. Recall that the scalar product on g is defined by {X,Y) ~ —b{X,Q{Yj). 
Hence 

(e,[77,ad(z)77]) = -6(e,e([77,ad(z),7])) 

= (ad(z)ad(0?7,f?) 

= (ad(z)ad(e)^','?') 
where C = ^ ■ with ^' € C{Joi('^) ^'^^ ij — k ■ rj' for some k G K. We can then check 
that the symmetric endomorphism ad(0)ad(^') : p — >■ p is negative definite when 
G C^oi(-^)- ^'^'^ Lemma is proved. □ 

If Jy is a A'- invariant almost complex structure on Y compatible with 17 y-, the 
last Lemma tells us that (— ad(2;), , Jy) is a AT-invariant almost complex structure 
on p X y compatible with ilpxY in a neighbourhood of Y. 

Let us fix Up, such that ip~^{Uf}) = Br x Vp where Vp is a neighbourhood of 
Zp in Y and B,. := {X £ p | ||X|| < r}. The almost complex structure Jm on 
defined by (p*{Jm) = (— ad(z),, Jy) is compatible with Hm if V;3 and Br are 
small enough. Finally we see that the symbol f*{c%f\u^) is equal to the product 
CTi cr2|B,xv^, where 

a2{X, y; i], v) = c{v - ki{X, y)), {X, y; i], v) £ T(p x Y), 
acts on ApTj,y ® Ly, and 

ai{X,y;7],v) = c{t] ~ K2{X,y)), {X,y;r],v) G T(p x Y), 
acts on App^ (here p^ denotes the complex X-module (p, — ad(2;))). 

Let Ky be the Kirwan vector field on Y associated to the moment map ■ 
We denoted Cy, the symbol Thom(y, Jy) ® Ly pushed by the vector field Ky. By 
definition Q^{Y) is the equivariant index of the if-transversally elliptic symbol 

The Atiyah symbol Atp on p is defined by the following relations : for {X, rj) e 
Tp, 

(36) At,{X,r,) :=c(7? + [z,X]) : Ar"p- ^ A^'^'ip-. 

Lemma 4.13. The symbols ai cr2|_BrxVf( Atp Cy|B^xv,3 define the same 
class m K%{TK{Br x V/j)). 

Proof. We consider the paths s G [0, 1] ^ A" := [e"-^ • $y (j/)]{, Kf{X,y) = Af,(2/), 
and k| {X, y) = — [A^ , X] . We define then the paths at the level of symbols : erf 
and (t|. We check that 

Char (crj 0cr|) nTK(y x p) = {{X,y;v,ri) \ v = A'yiy) = 0,and 77 = [A", X] = 0}. 

But since $y (y) S tj^, the condition [A'',X] = [e^ ■ (y) , X]p = forces X to be 
equal to 0. Hence we get 

CharK f^2) n Tk{Y x p) ~ Cr(||$^||2) x {0}, Vs G [0, 1]. 

We have proved that s G [0, 1] 1— >■ uf crUs^xv^ is an homotopy of transversally 
elliptic symbols: cti (J2 and ct° tTj define the same class in K^(T/f (i?^ x V/3)). 
We see that tr^ = ^y ^^d we have 

a°(X,y;,7,«) = c(7?+[i^),X]). 
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We consider another path of symbols 

T*{X,y;r,,v)^c{f^+[ti§iii) + {l~t)z,X]), tS [0,1]. 
We check that if {X, y; t], v) e Char(T* c^) n Tk{p x Y) then the vector t](Bv& 
T(jcy)(p X Y) is orthogonal to the vector field generated by $y(y) and we have 
moreover v = Kyiy) and i] = — (j/) + (1 — t)z,X]. Thus 

= \\KY{y)f+[H + il-t)z,X],[lx]) 

= \\KY{y)\\' + t\\[lx]\\' + il-t){[z,X],[lx]) 

<5 

where ^ = <&y (y) € K ■ C^^^{z). Since ^ is strongly elliptic and thanks to Lemma 
I4.12[ we know that the term 5 is strictly positive if X 7^ 0, thus Ky(y) = and 
X = 0. 

We have proved that t € [0,1] 1— >■ T*0Cy|B^xV^ is an homotopy of ii'-transversally 
elliptic symbols: (T2 ^^'^ © define the same class in K'j^{TK{Br x 
Vp)). □ 

At this stage we know that 

Q^M) = Indexf^.v^ {At,U © c^lv,) e R-^{K). 

Since Cy Atp is also isT-transvcrsally elliptic onVp xp, the excision property gives 
also 

Qi.{M) = Index^^'^.p (Atp c« |v,) e R-^{K). 

Let 5'"'^ be the circle subgroup of K with Lie algebra equal to M.z. We can consider 
p as a 5^ X if-manifold. We note that the Atiyah symbol Atp is 5^ x if-equivariant 
and S^-transversally elliptic. Its index is computed in [1], see also [34| [Section 5]. 
We have the following relation 

Indcxf ^^(Atp) = 5'(p) 

in R-°°{S^ X K). 

So we have two classes Atp e K|^i^^^(T5ip), and Cylvf, G (T^V^). By the 
multiplicative property (see Theorem l4.ip we know that their product Atp0Cy|v^ € 
■^SfxSi ('^^xS'^ ^ P)) following X A'-equivariant index 

Indexf (Atp c?^ I V, ) = Index^^ ><^'(Atp) Indexff^ (c?^ | ) 

= ^•(p)0lndex^);(cf.|v,) 
= 5'(p)0Q^(y) eR-°°iS^xK). 
Finally, thanks to the restriction property (|24p . we know that 

Q^(M) Indcxf.v, (Atp c?.|v,) G 
is equal to the restriction of 

Index^;,^f (Atp c?.|v,) = ^•(p) Q^,(r) £ i?-°°(5i x A') 
to the subgroup K ^ x K . The Theorem is then proved. □ 
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Remark 4.14. The Assumption Al is used because we don't how to prove the 
equality 

when the set B is not finite, e.g. the set Cr(|j$;^||^) is non-compact. 

4.5. Proof of Theorem I3.30i Here we work with a pre-quantized Hamiltoniaii 
if-manifold (P, Hp, $p), and we assume that the map ($p , z) is proper. Here 
is the Lie algebra of a circle subgroup C K contained in the center of K . 

We are in the context of Theorem 14.101 We have a decomposition t = ^2 
where ti := Rz and ^2 are ideals of t and the moment map (<i>p , z) relative to the 
S'^-action is proper. Then we have the following equality 

(37) Q^°° (A/) = Q| (P) = Q<^*'"> (P) e p-°° {K) 

where the right hand side is computed via a localization procedure on the set Cr((/3p) 
of critical points of the proper map ipp := {{^p, z)Y . We note that 

Cr(^p) = ^pl(0)UP^ 

We are interested in the following cases 

(1) P is a proper Hamiltonian G-manifold (A/, JIm, $^/) with a moment map 
taking values in G ■ C^^^{z), and which satisfies Assumption A2. 

(2) P is the symplectic slice Y of the former case M := G XkY ■ 

Thanks to Lemma 13.211 we know that in the two cases described above, the 
proper map ipp \s strictly positive : hence ip~^^{Q) = 0. Let us compute the gener- 
alized character Q^^'''\P) in this case. 

Let Hi^p be the Hamiltonian vector field of -^fp- The symbol Thom(P, Jp) ® 
Lp pushed by the vector field K,ip is denoted Cp. Let Bp the set of connected 
component of P^. For any X G Bp, we consider a relatively compact open K- 
invariant neighbourhood Z//;^' oi X such that Ct::{lpp)C]Ux = X. Wc denote Q'^{P) G 
R~°°{K) the equivariant index of the S'^-transversally elliptic symbol c'^ji^^. 

When </jp^(0) = 0, the generalized character Q^'^^(P) is defined by the relation 

(38) QP'Hp)^ J2 Qk{P)^R-^{K). 

xeBp 

For X e Bp, we denote 

• Lx the restriction of the Kostant-Souriau line bundle Lp on X , 

• Mx the normal bundle of X in P, and \MxY i^^'^ are the z-polarized 
versions (see Section |3^ . 

If we use ([57)) and ([55)1 . the proof of Theorem 13.301 is reduced to the following 
Proposition 4.15. We have the following equality in R^°°{K): 

(39) Q^(P) - (-l)'^-RR^ {X,Lx ® dct{Af+'') ® S'{\Afxn) , 
where rx is the complex rank ofM^'^. 

Proof. Relations ^ show that = {^p,z)zp. Since {^p,z) > in a neighbour- 
hood of Ux, we can replace K,p by the vector field zp without changing the index of 
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the corresponding transversally elliptic operator. This means that Qk{M) is equal 
to the index of cr'^\ux, where the symbol ct^ is defined by : for {m,v) E TP, 

(40) a'im, v) := c{v - zp{m)) : A^™"T„P ® Lp\^ A^'^'^T^P ® Lp\^. 

We have proved in [51] [Theorem 5.8], that the index of (7^\ux is equal to the right 
hand side of ((39l) . Hence the proof is completed. □ 

We want now to clarify the convergence of the sum that appears in (j38p . when 
is non-compact. 

Let r be a maximal torus in K: it contains the circle subgroup S^. Let A C t 
be the lattice which is the kernel of exp : t — >■ T. Let Zq G R^°z n A that generates 
the sub-lattice Rz D A: the torus acts on an irreducible representation 

through the character t t"- with n = e Z. We have then a graduation 

R{K) = X^nez ^n{K) where Rn{K) is the group generated by the representations 
such that = n. We see that i?„(if) • Rm{K) C Rn+m{K). 

For any n G Z, we denote R>„{K) (resp. Ry^{K)) the subgroup formed by the 
finite (resp. infinite) sum J2i>n^i where Ei G Ri{K). We have the following basic 
lemma 

Lemma 4.16. • If A € RZj^lK) and B € RZ.'^{K), then the product A ■ B is 
well-defined and belongs to i?>J^j„(A'). 

• An infinite sum J2n>o^"' ^ ^>^{^)' converges in Ry'^{K). 

Proof. The proof is left to the reader. □ 

For X e Bp, the action of is trivial on X, and relation ([TT|) shows that 5^ acts 
on the fibres of Kostant-Souriau line bundle Lx through the character t i— )■ 
where n{X) = ^^-^ -j^"*'^"^ is ^ strictly positive integer. 

Proposition 4.17. • The generalized character Q'^{P) belongs to R'^^^^^{K). 

• The sum X^A'gBp converges in R^'^{K). 

Proof. The generalized character Q'^{P) is equal to the sum {—^Y'^'^■' J2p>o -^p^ 
with 

Ep = RR^'^iX, Lx ® det(AA+^^) ® SP{\NxV)) e R{K). 

Since the group acts on the fibres of the polarized bundles Mx'^ and \NxY 
through characters with n > 0, we see that Ep £ R>n(X)+p{K). Hence Q'^{P) = 
h^r^''^ Ep>o Ep converges in i?>:^^)(K). 

For the second point we see that ^xeBp Qxi^) ~ Sri>o 'w'itii 

E QKiP)^R>n{K). 
n(X)=n 

The former sum is finite (and then well-defined) because the map ($p , Zq) is proper 
: for any C > 0, we have only a finite number of A" e Bp such that ($p (A"), Zo) < C. 
The last point is proved. 

□ 
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4.6. Proof of Theorem 13.251 under Assumption A2. The proof of Theorem 
13.251 in the case where Assumption A2 is satisfied, follows directly of the results 
of the Section applied to the following two cases: 

(1) (Af, rii/, is a proper Hamiltonian G-manifold with a moment map 
taking values in G ■ Cy^^^{z), and which satisfies Assumption A2. 

(2) y is the symplectic slice of the former case M :~ G Xj^ Y. 

Note that the fixed point set and coincide. For a connected component 
X oiY^, let Afx (resp. J^f^) be the normal bundle of X in M (rcsp. Y). Since the 
normal bundle of Y in M is the trivial bundle y x p, we have Afx = ^'x ® P- ^ 
small computation shows that 

^V" = {^'xY'' and |AA^|+^^ = |AA^|+'^®(p,ad(z)). 
Finally (f37| and Proposition [39l gives 
Qirm = ^(-l)'--RR^(A',L^®det(AA+^^)®5-(|A0,r)) 

X 

= Y.^-^y^'^^'' ('^' ® det(A/'!,)+'" ® ^'dA^;,!") (g> S'ip)) 

X 

We know that the term ^^(-1)'''^RR^ (A', Xa' «) det(A/'jf )+'^ ® S"(|7V^h')) be- 
longs to Ky^{K) (see Proposition liTf)) . We see also that S"(p) e Ky'^iK). 
Hence their product is well-defined (see Lemma [4.161 . 

5. Appendix: proof of Lemma [3.281 

Let m = X^^eGi 1(2) "^a'^/F ^ R^°°{G, z) that is G"-admissible : the map TTg'^g : 
G ■ Support(TO) (g')* is proper. We start with the 

Lemma 5.1. • There exists a closed subset Cm C C'(Jqj(z) such that 
TTg'^g (G • Support (to)) = G' ■ Cm- 

• The projection ttc^j,' : G' ■ Cm — > (?')* *s proper. 

• T/ie projection iTf^t ■ '^t,s)iG ■ Support(m)) — > (t')* is proper. 

Proof. First we note that G ■ Support(m) is closed in q*. Thanks to point (d) 
of Proposition 13.151 we know that 7rg/_g (G • Support(m)) = G' • Cm, with Cm — 
7rg'_g(G • Support(m)) n (t')* C C'^^i{z). The properness assumption forces G' • Cm 
(resp. Cm) to be closed in (g')* (rcsp. C'{Jq[(z)). The first point is proved. 

Let Br := G (t')* | ||^'|| < R}, and consider e^' • A' G G' • G,„ n 7r-^g,(Bj7). 
Since ||7r{'^g'(e"^ • A') > ||A'||, we see that A' belongs to a compact subset K, of 
C'^q[(z). Then there exists a constant c(/C) > such that ||7r{'^g'(e^ • A')|| > 
c(/C)||X'|p for any F € p' and A' G /C (see the proof of Proposition [^iO]) . Finally, 
we have proved that ■ A' belongs to a compact subset of G' • Cm- 

Let rj £ G ■ Support(TO) such that ^ = nt.g{ri) G ttjT {(Si?). Then 7r{'_{(^) = 
TTf^gir]) = 7r{',g' °7rg'.g(?7) G Thanks to the former point, we know that TTg'^girj) 
is bounded. Since TTg/.g : G • Support(m) — >■ (g')* is proper, it implies that 77 and 
£. = T^i,g{v) are bounded. 
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□ 



Consider now the restriction of m € i? °°{G,z) to K and G' . The generalized 
character n := tk.g{'^) = X^Aeifh ^ defined by the relations 

MeGhoi(2) 

and q Tccirn) = TlijeG' (z) l^^^i^ i defined by the relations 

MSGhol(2) 

We see that A G Support(n) only if there exists /i such that m^Q{{G ■ ii)\^k) ^ 0: 
hence 

Support(7i) C TTj^g (G • Support(m)) . 

Since ttc^j is proper on Trj^g (G • Support (m)), we have that Trc^t is proper on K ■ 
Support(n). Thus n g R~°°{K,z) is i^'-admissible. 

We will now compare the following two elements of R^°°{K' , z) : A := vk' .K{n) = 

E5ex;„,(.) and B := rK',G'{q) = ^s'^^/'- 

By definition, we have 

as = ^ »^aQ((A' • A)5,K') 

AGi?hoi(^) 

= E ™M E Q((G-A^)A,/^)Q((A-.AkA")h 
/^eGhoi(z) \Ae/?hoi(z) / 



X 



and 



6a- = 51 (lM{G' ■v)s^K') 

= E "Vl E 2((G'-^^kK')S((G-MkG') 

MeGhoi(2) \i^eGhoi(z) 



We have 



Aei?i„i(z) 
Aeifhoi(z) 

which is finite if 7^ 0. Similarly, we check that Y^^s = [V/*" : V'^jx']- 

Finally, we have proved that rK',K ° ^K,G{'n^) = A = B ^ ^k',G' ° rG',G(w)- 
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